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MOVING PICTURE CODING APPARATUS 



BACKGROUND OF THE INVENTION 

[0001] The present invention relates to a bit rate control method for a moving 
picture coding apparatus to produce the moving picture data in which the bit rate 
is different[[,]] from the compression coded moving picture data. 

[0002] Recently, according to the development of the digital signal processing 
engineering, the-moving picture data can be compressed and coded, and the 
compr e ssion compression and coded moving picture data is e as il y tr ea t e d easv to 
handle . Further, with the development of a computer network, the chanc o to 
transm i t and rocoivo tho comprossion compressed and coded moving picture 
data are increasingly transmitted and received through various transmission 
paths i s incr e as e d . For example, as a TV broadcasting station, a large amount of 
the moving picture data previously accumulated, is compression coded and 
saved, and as a VOD (Video On Demand), at the need of the user, the 
compr e ss i on compressed and coded moving picture data can be transmitted onto 
the transmission path. 

[0003] How e v e r, i n th e c a s e wh e r e th e compr o ssion coded moving pictur e 
data i s transmitted onto tho transm i ss i on path, there i s somot i mes a cas e wh e r e 
th e b i t rat e which can b e transm i tt e d on th e transmission path is d i ff e r e nt from 
th e b i t rat e of th e mov i ng pictur e , and in tho caso wh e r e tho bit rato of tho mov i ng 
pictur e data is larg e , wh e n th e mov i ng p i cture i s transmitted as it is, thoro is a 
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probl e m that th e d el ay is g e n e rat e d in th e r e c e ption data, and tho mov i ng p i ctur e 
dat a c a n not bo roproducod i n the r e a l timo. Accord i ng l y, i n ordor to r e produc e a 
mov i ng p i ctur e in th e r e al t i m e , i t is n e c e ss a ry that tho b i t rat e of th e mov i ng 
p i ctur e i s l ow e red. Furth e r, it is n e c e ssary that th e b i t rato of th e mov i ng p i ctur e 
data is adjust e d a l so to th e b i t rat e at wh i ch the rocoption term i na l oqu i pmont by 
which tho mov i ng picture data is rooeivod, can roco i vo. Further, ov e n in th e cas e 
wh e r e not the who l e fram e in th e compr e ss i on cod e d mov i ng p i cture data, but 
on l y a port i on of th e fram e i s quarried out and transm i tt e d, wh e n th e b i t rat e of 
th e quarr ie d out mov i ng pictur e data e xc ee ds th e b i t rato at which th e 
transm i ss i on path can transm i t, i t is nocossary that tho proc e ssing to l ow e r th e b i t 
rat e i s conduct e d. As d e scr i bed abov e , wh e n th e mov i ng picture data i s 
transm i tt e d through var i ous rocopt i on t e rmina l e quipments and transm i ssion 
paths, i t i s n e c e ssary that tho b i t rat e of th e moving p i ctur e data is chang e d by 
adjust i ng to various b i t ratos, and by a timo period i n wh i ch it takes for the 
processing of th e rate control, th e d el ay i s gen e rat e d in th e data transm i ss i on 
However where the compressed and coded moving picture data is transmitted to 
the transmission path, it is possible that the bit rate which can be transmitted on 
the transmission path differs from the bit rate of the moving picture. If the bit rate 
of the moving picture data is high, when the moving picture is transmitted at that 
rate, there is a problem in that delay is generated in the reception data, and the 
moving picture data cannot be reproduced in real time. Accordingly, in order to 
reproduce a moving picture in real time, it is necessary that the bit rate of the 
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moving picture be reduced. Further, it is necessary that the bit rate of the moving 
picture data also be adjusted to the receiving bit rate of the terminal eguipment 
receiving the moving picture data. Further even in the case where not the whole 
frame in the compressed and coded moving picture data, but only a portion of the 
frame is selected and removed and transmitted, when the bit rate of the selected 
and removed moving picture data exceeds the bit rate of the transmission path, 
processing must be conducted to the reduced bit rate. As described above, when 
the moving picture data is transmitted through various reception terminal 
equipment and transmission paths, it is necessary that the bit rate of the moving 
picture data be changed by adjusting to various bit rates, and a delay is 
generated in the data transmission by a time period for the processing of the rate 
control . 

[0004] Further, when the VOD server sends out a plurality of moving picture 
data to a plurality of terminal equipment[[s]L as in the case of the VOD, m 
proport i onal to th e p l ura li ty of number of term i nal equipments connoctod to th e 
VOD s e rv e r, b e caus e th e rat e contro l proc e ssing bocom e s h e avy, and th e 
burd e n onto th e VOD s e rv e r b e com e s l arg o , tho number of termina l equ i pm e nts 
which can b e s i mu l tan e ously conn e ct e d, ar e the Quantity of simultaneous 
connected terminal equipment is limited. This is because excessive rate control 
processing is required, burdening the VOD server. That is, the bit rate control 
method [[toll that quickly changes the bit rate of the moving picture data is 
absolutely necessary. 
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[0005] In such tho cas e , as th e conv e nt i onal t e chn i que to contro l th e b i t rat e 
of tho compress i on cod e d mov i ng pictur e data, a m e thod i n wh i ch, in i t i a ll y, th e 
mov i ng pictur e data i s d e cod e d i nto th e non - compr e ss i on mov i ng pictur e d a ta, 
and cod e d aga i n, and th e r e by th e b i t rat e i s chang e d, i s wo l l known. How e v e r, i n 
th i s method, b e caus e th e moving p i ctur e d a ta is d e coded onc e , and furth e r, 
cod e d aga i n, th e burdon of th i s proc e ss i ng i s l arg e , and th e r e is a prob le m that it 
i s d i fficu l t to quick l y chang e th e b i t rat e and produce the mov i ng p i ctur e data The 
conventional and well known technioue of controlling the bit rate of compressed 
and coded moving picture data is to initially decode the data into non- 
compressed moving picture data and then to code it again, thereby changing the 
bit rate. However in this method, excessive processing is reguired because the 
moving picture data is decoded and coded again. In addition, it is difficult to 
quickly change the bit rate and produce the moving picture data . 

[0006] Further, as the conventional technique to lighten the re-coding 
processing and to increase the processing speed, a technique disclosed in JP-A- 
8-23539 is well known. In FIG. 38, the structure of the conventional moving 
picture coding apparatus is shown. In FIG. 38, a moving picture coding apparatus 
5001 is structured by a variable length decoding means 5002 connected to an 
input means 5006, a re-quantizing means 5003, a variable coding means 5004, a 
buffer memory means 5005, and a buffer occupation amount detecting means 
5006, and is connected to an output means 5008. 
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[0007] Next, an operation of the moving picture coding apparatus will be 
described. In FIG. 38, the input means 5007 inputs the coded moving picture 
data into the variable length decoding means 5002 for each one frame, and 
inputs the ob ie ct i v e desired bit rate into the re-quantization means 5003. Then, 
the variable length decoding means 5002 conducts variable length decoding on 
the input data, and the quantized DCT (Discrete Cosine Transform) coefficient is 
found, and outputted to the re-quantization means 5003. Then, the re- 
quantization means 5003 re-quantizes the quantized DCT coefficient, and 
outputs it to the variable length coding means 5004. In this connection, the re- 
quantization means 5003 compares the bit rate inputted from the input means 
5007 to the buffer occupat i on fullness amount inputted from the buffer occupation 
amount detecting means 5006, and the quantization value is set so as to satisfy 
a predetermined bit rate, and the re-quantization is conducted. Herein, the 
quantization value means a value to divide the DCT coefficien t is divided in the 
quantization. Further, the variable length coding means 5004 conducts variable 
length coding on the re-quantized DCT coefficient, and supplies the moving 
picture data which is variable length coded, to the buffer memory means 5005. 
The buffer memory means 5005 outputs the inputted moving picture data from 
the variable length coding means 5004 to the output means 5008, and outputs 
the data amount of the moving picture data to the buffer occupation amount 
detecting means 5006. After the buffer occupation amount detecting means 5006 
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adds the data amount and detects the buffer occupation amount, the total 
amount of the data is outputted to the re-quantization means 5003. 

[0008] As described above, when, by using the moving picture coding 
apparatus 5001, the bit rate is controlled from the compr e ss i on compressed and 
coded moving picture data and the new moving picture data i s now l y produced, 
the moving picture data is produced through the process in which the input 
moving picture data is once variable length decoded, re-quantized, and variable 
length coded. That is, because the moving picture data is decoded and coded 
again [[to]]until the inverse quantization processing, the calculation load is large 
and i t can b e said that it is difficult to quickly conduct the rate control. Further, the 
moving picture data is structured by a plurality of frames, and because, when the 
moving picture data is compr e ssion compressed and coded, in order to increase 
the efficiency, g e n e ral l y, th e corre l at i on with th e pr e c e d i ng fram e by on e fram e i n 
t i m e is used, and the interframe predictive coding , which is the process of using 
the correlation between the given frame and the immediately preceding frame, is 
generally b e tw ee n fram e s i s used. Then, when the moving picture data includes 
the frame on which [[thell interframe predictive coding b e tw ee n fram e s is 
conducted, [[the]]a problem exists when the re-quantization is conducted by 
using the moving picture coding apparatus. 

[0009] Wh e n jn the interframe predictive coding b e tw ee n frames i s used, the 
frame (Pi) on which [[the]] re-quantization is conducted in the moving picture 
coding apparatus 5001 is conducted , is used for us i ng for th e predict i v e cod i ng 
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b e tw ee n fram e s i n th e subs e qu e nt fram e (P i +1) aft e r on e fram e i n t i m e , and is 
th e fram e which is n e c e ssary for add i tion wh e n th e (P i +1) i s d e cod e d when it is 
timewise at (Pi+1), which is the frame immediately following Pi. The frame Pi is 
necessary for decoding the (Pi+1 ) frame. 

[0010] Then, when the re-quantization is conducted in the moving picture 
coding apparatus 5001, because the re-quantization means changes the 
quantization value of the input data, the frame (Pi) b e for e th e prior to re- 
quatization is conduct e d, differs from the frame (Pi 1 ) after the-re-quantization-is 
conduct e d . Accordingly, when the-Pi' which is chang e d by tho ro quant i zation, is 
added to the-(Pi+1) on which th e i nv e rs e quant i zat i on i s conducted, and the 
(Pi+1) is decoded, the decoded image is deteriorated because the difference 
exists between the Pi to be originally added and Pi' , th e decoded i mag e is 
d e t e riorat e d . Hereinafter, the difference between the Pi and Pi' is called as the 
motion compensation error. That is, to th efor moving picture data feron which 
the inter-frame predictive coding betwe e n fram e s is us e d has been conducted , 
when the moving picture data is produced by using the moving picture coding 
apparatus 5001, the-image quality deterioration is caused due to the motion 
compensation error. Further, in order to prevent the image quality deterioration, it 
is necessary that the frame next to the frame on which the re-quantization is 
conducted, is re-corded including the motion compensation, and there is a 
problem that the processing time is further increased. 
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[0011] In the conventional moving picture coding apparatus, when the bit rate 
of the moving picture data which is previously coded, is changed, and the new 
moving picture data is newly produced, it is necessary to code again after the 
moving picture data is decoded once and re-quantized again O. Accordingly, 
aftd-it is difficult to quickly produce the moving picture data. 

[0012] When the conventional moving picture coding apparatus is used and 
the rate is controlled by conduct i nq performinq the re-quantization, it is difficult to 
conduct the rate control without causing the image quality deterioration in the 
next frame on which the re-quantization is conductedfM I. This is booauso tho 
i mago qua li ty dotor i orat i on due to toe-motion compensation erro r i s caused, i t i s 
d i fficu l t to — conduct — the — rate — contro l — without — c a us i ng — the — i mag e — qua li ty 
det e r i orat i on . 

SUMMARY OF THE INVENTION 

[0013] An object of the present invention is to solve the above mentioned two 
problems. That is, the object is that: when the bit rate of the coded moving 
picture data is changed, and the moving picture data is newly produced, the 
moving picture data production is quickly realized without decoding the moving 
picture data, and without causing the image quality deterioration due to the 
motion compensation error. 

[0014] In order to solve the above problems, in a moving picture coding 
apparatus in which the previously compr e ss i on compressed and coded moving 
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picture data is an input, and the new moving picture data whose bit rate is 
different[[,]] is produced and outputted, the present invention is structured as 
follows. 

[0015] F i rst l y, th e mov i ng pictur e data producing apparatus to produc e th e 
mov i ng p i ctur e data which i s an i nput of th e moving p i ctur e data cod i ng 
apparatus, i s prov i d e d with a produc i ng m e ans for a rate corr e ction data i n wh i ch 
tho b i t amount is d i ff e r e nt to an ar e a i n which th e b i t g e n e ration amount in th e P 
fram e (pr e d i ct i v e cod i ng imago botwoen fram e s) i s l arg e , and wh i ch can conduct 
th e rat e chang e . Firstly, a moving picture data producing apparatus, which 
creates moving picture data as an input for a moving picture coding apparatus, 
comprises a means to create a rate correction data with a different bit amount 
and a variable bit rate for the areas where the bit amount of a P-frame 
(interframe predictive coding picture) is large. 

[0016] According to the above description, when ffthel lnew moving picture 
data is n e w l y produced from the previously coded moving picture data, by 
adjusting to the objective bit rate, and by selecting and replacing the rate 
correction data whose bit amount is different in the input moving picture data, the 
moving picture data production can be quickly conducted by changing the bit rate 
without decoding the moving picture data. Further, by producing the rate 
correction data in an area in which the bit amount is large, the bit rate change 
can be effectively conducted. 
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[0017] S e condly, — the — moving — p i ctur e — data — produc i ng — apparatus — wh i ch 
produc e s th e moving p i ctur e data which i s a n i nput of the mov i ng p i ctur e coding 
app a r a tus i s prov i d e d w i th a m e ans by which th e rat e corr e ction data i n wh i ch 
tho b i t amount is d i fferent and th e rat e chang e can b e conducted, i s produced, to 
an ar e a which i s prev i ous l y determ i ned, and whoso probabi li ty to bo roforrod to 
from tho next frame wh e n th e mot i on est i mat i on i s conduct e d, i s l ow, i n th e P 
fram e (pr e dict i v e cod i ng i mago b e tw ee n fram e s). Secondlv. the moving picture 
data producing apparatus comprises a means to create rate correction data with 
a different bit amount and a variable bit rate for the predetermined areas, which 
have a low likelihood of reference (low probability of being referred to) from the 
next frame for motion prediction within the P-frame. 

[0018] According to this, wh e n th e new moving picture data i s n e w l v can be 
produced from [[the]] previously coded moving picture dataff.1 1. This can be 
accomplished by selecting and replacing the rate correction data whos e b i t 
amount i n th e i nput moving p i cture data i s d i fferent, by adjust i ng to tho object i ve 
b i t rat e . according to an objective bit rate. ffthell The bit rate is changed and the 
moving picture data can be quickly produced without decoding the moving 
picture data. Further, by producing the rate correction data [[in]]fgr an area 
whos e probab ili ty to b e referr e d to having a low likelihood of reference from the 
next frame (low probability of being referred to), a reduction o f . is low, tho 
l ower i ng of the predictive coding efficiency due to the influence of the search 
range limitation of the motion estimation can be reduced. 
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[0019] Th i rd l y, th e mov i ng pictur e d a ta producing apparatus to produc e th e 
mov i ng p i ctur e data which is an i nput of th e mov i ng p i ctur e cod i ng app a ratus, i s 
provid e d w i th a moans f or producing tho rato correct i on data whoso b i t amount 
i s d i ff e r e nt, i n th e P frame, and prov i d e d with th e mot i on comp e nsat i on m e ans for 
conduct i ng th e mot i on comp e nsat i on w i thout r e f e rr i ng to from tho area hav i ng tho 
rato correction data, wh e n th e mov e m e nt of tho next fram e i s for e cast. Thirdlv. 
the moving picture data producing apparatus comprises a means to create a rate 
correction data in the P-frames having a different bit amount from the original bit 
amount, and a motion compensation means for conducting motion compensation 
without referring to the areas that include the rate correction data in the motion 
prediction for the next frame. 

[0020] Accordingl y to th i s , when the new moving picture data is n e w l y 
produced from the previously coded moving picture data, even when the moving 
picture data is produced by selecting the rate correction data whose bit amount is 
different, in the input moving picture data, because the area is not subject to the 
motion estimation, the generation of the motion compensation error due to the 
replacement of the data can be prevented. 

[0021] Fourthly, th e mov i ng p i ctur e data produc i ng apparatus to produc e th e 
mov i ng p i cture data wh i ch i s th e i nput data of th e mov i ng picture cod i ng 
apparatus, i s provided w i th a m e ans for produc i ng th e rat e corr e ction data whose 
b i t amount i s d i ff e r e nt, and which can chang e th e b i t rate, to an aroa i n which the 
roforrod degree from th e n e xt fram e wh e n th e mov e m e nt is forecast, i s l ow, i n 
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th e P fr a m e . Fourthlv. the moving picture data producing apparatus is provided 
with a means for producing the rate correction data whose bit amount is different, 
for an area having a low likelihood of reference from the next frame during 
motion prediction within the P-frame. In addition, the moving picture data 
producing apparatus can change the bit rate for such an area. 

[0022] According}^ to tho above descr i pt i on, whon tho mov i ng p i cture data 
produc i ng apparatus produc e s th e mov i ng pictur e data, th e moving picture data 
can be produced without l ow e r i ng reducing the coding efficiency of the interframe 
predictive coding b e tw ee n fram e s . 

[0023] F i fth l y, th e mov i ng p i ctur e dat a produc i ng apparatus to produc e th e 
mov i ng p i ctur e data wh i ch i s the i nput data of th e mov i ng picture cod i ng 
apparatus is provided w i th a m e ans for produc i ng th e rat e corroct i on data, by 
remov i ng tho high froquoncy component of tho orig i na l i mag e , and by conduct i ng 
th e pr e d i ct i ve cod i ng b e tw e en framos, to oach P fram e of th e mov i ng p i ctur e 
datar Fifthlv, the moving picture data producing apparatus is provided with a 
means for producing the rate correction data, by removing the high freguency 
component of the original image, and by conducting the predictive coding 
between frames, for each P-frame of the moving picture data. 

[0024] Accord i ng to tho above descr i ption, whon tho moving p i ctur e data i s 
n e w l y produc e d w i thout decod i ng the previous l y coded mov i ng p i ctur e data, 
correspond i ng to th e obj e ctiv e b i t rat e , by s ele ct i ng oach aroa of th e rat e 
correction data cod e d by r e mov i ng tho high fr e qu e ncy component, tho f i ne b i t 
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rate contro l can bo qu i ck l y conduct e d. According to the above description, when 
new moving picture data is produced without decoding the previously coded 
moving picture data, fine bit rate control can be guicklv conducted according to 
the desired bit rate by selecting each area of the rate correction data coded by 
removing the high frequency component. 

[0025] Sixthly, the moving picture data producing apparatus to produc e th e 
mov i ng p i ctur e data which is th e input data of tho moving pictur e cod i ng 
apparatus is provided with a means for producing to create the area information^ 
which identify deletable parts in the back part o f show i ng an ar e a of th e r e ar 
portion at wh i ch tho b i t can bo do l otod, i n each area, as the rate correction data. 

[0026] According to the above description, in the moving picture coding 
apparatus which produces the new moving picture data from the coded moving 
picture data, by adjust i ng to th e obj e ct i v e b i t rat e , wh e n e ach area of tho input 
mov i ng pictur e data is s e l e ct e d, and th e r e ar port i on b i t i s d ele t e d, th e b i t rat e 
contro l can bo qu i ckly conductod bit rate control can be guicklv conducted 
according to the desired bit rate, when each area of the input moving picture data 
is selected, and the rear portion bit is deleted . 

[0027] Seventhly, the moving picture data producing apparatus to produc e th e 
mov i ng picturo data wh i ch i s th e input data of th e mov i ng p i cture coding 
a pparatus is provided with a means for produc i ng th e creating an Lframe whos e 
b i t amount is d i ff e r e nt, as th e rat e correct i on data, to for each P-frame as a rate 
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correction data, wherein each l-frame has a different bit amount corresponding to 
the respective P-frame . 

[0028] Accord i ng to th i s, i n th e mov i ng p i ctur e cod i ng apparatus wh i ch 
produces th e n e w mov i ng picture data from th e pr e v i ous l y codod mov i ng p i cture 
data, by adjust i ng to tho object i v e b i t rat e , and by rep l ac i ng P framo of tho i nput 
mov i ng p i ctur e data with th e I framo whoso b i t amount i s d i ff e r e nt wh i ch i s th e 
rate corr e ct i on data, th e b i t rato control can bo qu i ckly conducted w i thout 
d e cod i ng th e input mov i ng pictur e data. Accordinglv. in the moving picture coding 
apparatus, bit rate control can be guicklv conducted according to the desired bit 
rate without decoding the input moving picture data, by replacing the P-frame of 
the input moving picture data with the l-frame whose bit amount is different and 
which is the rate correction data. 

[0029] E i ghth l y, i n tho caso whoro the mov i ng picture data producing 
apparatus to produc e th e moving picture data wh i ch i s tho input of th e moving 
p i cture coding apparatus is provided with a moans for dotormin i ng the quarry out 
ar e a i n e ach fram e , and for produc i ng th e rat e corr e ct i on data by which th e r a t e 
corr e ction i s poss i b le , for at le ast an ar e a mor e than one i n e ach quarry out aroa 
i n th e frames, and a mot i on compensat i on moans for i nhib i ting th e motion 
e st i mat i on outs i d e tho ar e a hav i ng th e rat e corr e ct i on data in th e pr e c e ding 
framo at th e t i mo of motion comp e nsation and th e quarry out aroa, whon ono 
port i on i n tho framo is quarr i ed out from th e pr e v i ous l y codod mov i ng p i ctur e data 
and th e moving p i cture data is newly produc e d, corr e spond i ng to tho object i v e bit 
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rat e , by s ele ct i ng tho data whos e b i t a mount i s d i ff e r e nt, th e bit rat e i s contro lle d 
and th e mov i ng p i ctur e data can b e qu i ck l y produc e d w i thout d e coding th e 
mov i ng p i ctur e data. Furth e r, b e caus e th e mot i on e st i mation to th e outsid e of th e 
quarr ie d out ar e a i s not conductod, ovon by using on l y tho quarr i od out area of a 
port i on of th e framo, tho d e cod i ng can b e conduct e d without g e n e rating th e 
mot i on — componsat i on — efref rEighthlv, the moving picture data producing 
apparatus produces moving picture data. Said moving picture data is an input of 
the moving picture coding apparatus. The moving picture data producing 
apparatus is provided with a means for: determining the select and remove area 
in each frame; and for producing the rate correction data by which the rate 
correction is possible for at least more than one area in each selected and 
removed area in the frames. The moving picture data producing apparatus is 
further provided with a motion compensation means for inhibiting the motion 
estimation outside of the areas having the rate correction data in the preceding 
frame at the time of motion compensation and the select and remove areas. 
When one portion in the frame is selected and removed from the previously 
coded moving picture data and the new moving picture data is produced 
according to the desired bit rate, by selecting the data whose bit amount is 
different, the bit rate is controlled and the moving picture data can be guicklv 
produced without decoding the moving picture data. Further, because motion 
estimation is not conducted outside of the selected and removed area, even by 
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using only the selected and removed area of a portion of the frame, the decoding 
can be conducted without generating the motion compensation error. 

BRIEF DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0030] FIG. 1 is a block diagram showing the structure of a moving picture 
data producing apparatus in the first embodiment of the present invention; 

[0031] FIG. 2 is a block diagram showing the structure of a moving picture 
coding apparatus in the first embodiment of the present invention; 

[0032] FIG. 3 is a block diagram showing the structure of the moving picture 
data producing apparatus in the second embodiment of the present invention; 

[0033] FIG. 4 is a block diagram showing the structure of the moving picture 
data producing apparatus in the third embodiment of the present invention; 

[0034] FIG. 5 is a block diagram showing the structure of the moving picture 
data producing apparatus in the fourth embodiment of the present invention; 

[0035] FIG. 6 is a block diagram showing the structure of the moving picture 
data producing apparatus in the fifth embodiment of the present invention; 

[0036] FIG. 7 is a block diagram showing the structure of the moving picture 
coding apparatus in the fifth embodiment of the present invention; 

[0037] FIG. 8 is a block diagram showing the structure of the moving picture 
data producing apparatus in the sixth embodiment of the present invention; 

[0038] FIG. 9 is a block diagram showing the structure of the moving picture 
coding apparatus in the sixth embodiment of the present invention; 
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[0039] FIG. 10 is a view showing an example of an area dividing the inside of 
the frame in the first embodiment of the present invention; 

[0040] FIG. 1 1 is a view showing a compression frame buffer structure in the 
first embodiment of the present invention; 

[0041] FIG. 12 is a view showing the rate correction area data in the first 
embodiment of the present invention; 

[0042] FIG. 13 is a view showing the compression frame data in the first 
embodiment of the present invention; 

[0043] FIG. 14 is a view showing the moving picture data structure in the first 
embodiment of the present invention; 

[0044] FIG. 15 is a view showing the data structure of the rate correction data 
in the first embodiment of the present invention; 

[0045] FIG. 16 is a view showing the content of the rate correction data in the 
first embodiment of the present invention; 

[0046] FIG. 17 is a view showing the rate correction data header structure in 
the first embodiment of the present invention; 

[0047] FIG. 18 is a view showing the referred area data in the third 
embodiment of the present invention; 

[0048] FIG. 19 is a view showing the compression frame data in the fourth 
embodiment of the present invention; 

[0049] FIG. 20 is a view showing the rate correction data header structure in 
the fourth embodiment of the present invention; 

Substitute Specification, Marked-up copy Page 17 of 91 



Serial No.: 09/899,907 
Filed: July 5, 2001 
Conf.No.:3598 
Atty. Docket No.: 33782 

[0050] FIG. 21 is a view showing the structure of the moving picture coding 
apparatus in the fourth embodiment of the present invention; 

[0051] FIG. 22 is a view showing a flow of a rate control method in the fourth 
embodiment of the present invention; 

[0052] FIG. 23 is a view showing the structure of a V i doo Pack e t video packet 
in the fifth embodiment of the present invention; 

[0053] FIG. 24 is a view showing the Vid e o Pack e t video packet termination 
data structure in the fifth embodiment of the present invention; 

[0054] FIG. 25 is a view showing the rate correction data structure and the 
rate correction data header structure in the fifth embodiment of the present 
invention; 

[0055] FIG. 26 is a view showing the data content of the rate correction data 
in the fifth embodiment of the present invention; 

[0056] FIG. 27 is a view showing the rate correction data structure and the 
rate correction data header structure in the sixth embodiment of the present 
invention; 

[0057] FIG. 28 is a view showing the data content of the rate correction data 
in the sixth embodiment of the present invention; 

[0058] FIG. 29 is a view showing the structure of the moving picture data 
producing apparatus in the seventh embodiment of the present invention; 

[0059] FIG. 30 is a view showing an example of a quarry out area in the 1 
frame in the seventh embodiment of the present invention; 
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[0060] FIG. 31 is a view showing the rate correction area data structure in the 
seventh embodiment of the present invention; 

[0061] FIG. 32 is a view showing the compression frame data structure in the 
seventh embodiment of the present invention; 

[0062] FIG. 33 is a view showing the rate correction data structure in the 
seventh embodiment of the present invention; 

[0063] FIG. 34 is a view the rate correction data header structure in the 
seventh embodiment of the present invention; 

[0064] FIG. 35 is a view showing the structure of the moving picture coding 
apparatus in the seventh embodiment of the present invention; 

[0065] FIG. 36 is a view showing the Video Packet structure data in the fifth 
embodiment of the present invention; 

[0066] FIG. 37 is a view showing the structure of l z frame coding means in the 
sixth embodiment of the present invention; and 

[0067] FIG. 38 is a block diagram showing the structure of the moving picture 
coding apparatus which is the conventional technology. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0068] By using FIG. 1 to FIG. 37, embodiments of the present invention will 
be described below. In this connection, the present invention is not limited to 
these embodiments, but in the range not departing from the spirit of the 
invention, the present invention can be conducted in various modes. 
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[0069] Embodiment 1 

[0070] In the first embodiment, a moving picture coding apparatus by wh i ch 
the for performing data rate control i s conduct e d without decoding by us i ng th e 
previously compr e ss i on compressed and coded moving picture data, and the new 
moving picture data i s now l y produced, and its method will be described below. 

[0071] Initially, the moving picture data producing apparatus which previously 
produces the moving picture data which is an input of the moving picture coding 
apparatus, will be described below. 

[0072] Ifi-FIG. 1[[,]]shows the structure of the moving picture data producing 
apparatus to prev i ous l y produce the moving picture data having the data 
structure to quickly conduct the rate control , is shown . 

[0073] In FIG. 1, a moving picture data producing apparatus 101 is provided 
with[[:]] a frame input means 117 connected to an input means 116; motion 
compensation means 102; DCT conversion means 103; quantizing means 104; 
variable length coding means 105; inverse quantizing means 106 to conduct the 
decoding; inverse DCT conversion means 107; and a frame memory 108 to store 
the decoded frame ;, and i s prov i d e d w i th: a maximum bit amount area detecting 
means 110 which is connected to the variable length coding means 105 and 
successively detects an area having the maximum bit amount; reference 
inhibition area memory means 109; compression frame buffer 112 connected to 
the DCT conversion means 103; quantization means 111 which is connected to 
the compression frame buffer 112 and conducts the quantization; and variable 
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length coding means 113 ;, and i s prov i d e d w i th: a compression frame data 
combination means 114 which is conn e ct e d combines the moving picture data is 
connected to the variable length coding means 105, reference inhibition area 
memory means 109 and variable length coding means 113.. , and comb i n e s w i th 
the The moving picture data producing apparatus 101 . and has th e structur e to 
teejs connected to an output means 115. 

[0074] The operation of the thus structured moving picture data producing 
apparatus will be described below. 

[0075] Initially, the input means 116 inputs the non-compression image into 
the frame input means 117. When the frame input means 117 receives the frame 
coding end signal inputted from the compression frame data combination means 
114, the non - compr e ssion on e frame data i nputs non-compressed data for a 
frame is inputted into the motion compensation means 102. However, when the 
first frame data is inputted, it is i ndep e nd e nt from not related to the frame coding 
end signal, and simultaneously when the data is inputted from the input means 
116, the non comprossion non-compressed frame data is inputted into the motion 
compensation means 102. 

[0076] Th e n, wh e n th e mot i on comp e nsation m ea ns 102 conducts th e 
pr e dictiv e cod i ng botwo o n framos on the non compress i on framo data inputt e d 
from tho framo input moans 117, aft e r th e ar e a whos e corre l at i on i s high in th e 
pr e c e d i ng fram e by on e fram e i nputted from th e fram e memory 108 is dotoct e d, i t 
conducts tho subtract i on, and outputs tho subtract e d framo data to tho DCT 
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conv e rs i on m e ans 103. I n th i s cas e , th e mot i on comp e nsation m e ans 102 do e s 
not conduct th e motion d e t e ction from tho roforonc e i nh i b i t i on aroa of tho 
proc o d i ng framo by on e fram e inputt e d from th e r e f e r e nc e i nh i bit i on ar e a 
m e mory m e ans 109. Further, i t do e s not conduct th e mot i on comp e nsat i on on 
th e data on which tho cod i ng in th e fram e i s conducted, and tho i nput data i s 
outputtod to th e DCT conv e rs i on m e ans as it i s. Then, the motion compensation 
means 102 conducts the interframe predictive coding on the data it has received. 
It does this by detecting the area whose correlation is high in the immediately 
preceding frame. The immediately preceding frame is inputted from the frame 
memory 108. It then performs a subtraction, and outputs the subtracted frame 
data to the DCT conversion means 103. In this case, the motion compensation 
means 102 does not perform a motion detection from the reference inhibition 
area of the preceding frame, which is inputted from the reference inhibition area 
memory means 109. Further it does not perform a motion compensation on the 
data on which the coding in the frame is conducted, and the input data is 
outputted to the DCT conversion means 103 unchanged. 

[0077] The DCT conversion means 103 conducts the DCT conversion on the 
frame data wh i ch i s i nputt e d provided by the motion compensation means 102^ 
and outputs tho The DCT conversion means 103 then outputs the DCT 
coefficient to the quantization means 104 and the compression frame buffer 112. 

[0078] I n F I G. 10, an e xample of th e d i vid i ng method of an aroa in tho framo 
is shown. Horoin, tho area shown in FIG. 10 i s structur e d by an arb i trary numb e r 
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of macro b l ocks (for e xamp le , 16.tim e s.16 pix e ls), a nd any shap e mayb e 
a ll owab le , and i s not lim i t e d to th e shap e shown in F I G. 10. Furth e r, th e structur e 
of tho compress i on fram e buff e r i s shown i n F I G. 11. Th e compr e ss i on fr a m e 
buff e r r e sp e ct i v e ly cont i nuous l y accommodat e s th e quantizat i on amount and th e 
DCT co e ff i c ie nt, corr e spond i ng to e ach ar e a in F I G. 10. FIG. 10 shows an 
example of a method of dividing an area in a frame. The area shown in FIG. 10 is 
structured by an arbitrary number of macro-blocks (for example, 16x16 pixels). 
However, any shape may be allowable, and the area is not limited to the shape 
shown in FIG. 10. Further, the structure of the compression frame buffer is 
shown in FIG. 11. The compression frame buffer continuously accommodates 
the quantization amount and the DCT coefficient corresponding to each area in 
FIG. 10. 

[0079] The quantization means 104 quantizes the DCT coefficient obtained by 
the DCT conversion means 103 for each area shown in FIG. 10[[, and thell . The 
quantized DCT coefficient is then outputted to the inverse quantization means 
106 and the variable length coding means 105, an dwhile the quantization value 
used for the quantization is outputted to the compression frame buffer 112. 
Th eAs shown in Fig. 11, the compression frame buffer 112 makes the DCT 
coefficient for oho framo inputted by tho DCT convers i on moans 103 and the 
quantization value for each area shown in FIG. 10 , and th e quantizat i on valu e 
i nputted by the quant i zat i on m e ans 10 4 r e sp e ct i v el y correspond to each other-as 
shown i n FIG. 11 , and then stores them. 
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[0080] The variable length coding means 105 conducts the variable length 
coding on the quantized DCT coefficient, and outputs it to the maximum bit 
amount area detecting means 110 and the compression frame data combination 
means 114. Herein, the data which is coded by the variable length coding means 
105 is called the normal frame data. 

[0081] Further, the inverse quantization means 106 conducts th e performs an 
inverse quantization on the quantized DCT coefficien t quant i z e d — by th e 
quant i zat i on moans 104 , and outputs ttthe result to the inverse DCT conversion 
means 107. The inverse DCT conversion means 107 conducts th e performs an 
inverse DCT conversion on the DCT coefficient obta i ned in provided by the 
inverse quantization means 106, and outputs it to the motion compensation 
means 102. The motion compensation means 102 decodes the frame by using 
the inverse DCT converted coefficient and the decoding frame^ which is the 
immediately preceding frame by one frame inputted by the frame memor y, and 
r e n e ws th e d e cod i ng framo of th e fram e m e mory . In addition, the motion 
compensation means 102 updates the decoding frame stored in the frame 
memory. However, for the l^frame, the inverse DCT converted frame is stored in 
the frame memory as-iUs unchanged . 

[0082] As doscr i bod abovo, i n tho variable l ongth cod i ng m e ans 105, wh e n 
th e cod i ng of on e fram e i s comp le t e d, th e max i mum b i t amount d o tocting m ea ns 
110 d e t e cts a pr e d e term i ned numb e r of areas i n ord e r of ar e a hav i ng th e larg e r 
b i t amount from th e ar e a i nclud i ng th e max i mum b i t amount, i n th e fram e cod e d 
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by th e var i ab le le ngth cod i ng m e ans 105, and outputs th e rat e corr e ct i on ar ea 
data show i ng th e d e t e ct e d ar e as to th e r e f e r e nc e i nh i b i t i on are a m e mory m e ans 
109 and th e compr e ss i on fram e buff e r 112. In F I G. 12, an e xampl e of th e rat e 
corr e ct i on ar e a data i s shown. I n FIG. 12, a black aroa shows tho solocted areas 
by tho framo max i mum bit aroa dotocting moans, and this ar e a i s d e fin e d as th e 
r o foronco i nh i bit i on ar e a. A s described above, when the coding for one frame is 
completed, the maximum bit amount area detecting means 110 detects a 
predetermined number of areas from the encoded framesfencoded by the 
variable length coding means 105) in the order of bit amount starting from the 
maximum, and then outputs the rate correction area data, which indicates the 
detected area, to the reference inhibition area memory means 109 and the 
compressed frame buffer 112. FIG. 12 shows an example of the rate correction 
area data. In FIG. 12, shaded areas indicate the areas selected by the frame 
maximum bit area detecting means, and this area is defined as the reference 
inhibition area. 

[0083] The reference inhibition area memory means 109 outputs the rate 
correction area data show i ng th e d e t e ct e d ar e a by th e maximum b i t amount ar e a 
d e t e cting m e ans 110 to the motion compensation means 102 and the moving 
eieture compression frame data combination means 114. Then, the compression 
frame buffer 112 cuts out the DCT coefficient and the quantization value ffofllfor 
the area-corresponding area, from within the i ns i do of th e compression frame 
buffer, [[to]]for the reference inhibition area of the correction area data , which is 
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inputted by the maximum bit amount area detecting means 110, and outputs 
them to the quantization means 111. The quantization means 111 conducts 
the performs a quantization by using a p l ura l ity of quant i zation va l ue s before and 
aftor tho quantization va l uo s inputted from tho compress i on framo buffer 1 12, on 
tho DCT cooffic i ont i nputted from th e compr e ss i on framo buff e r 112, and outputs 
i t to th e var i ab l e le ngth cod i ng m e ans 113. on the DCT coefficient inputted from 
the compression frame buffer 112 and outputs it to the variable length coding 
means 113. It does this by using a plurality of guantization values before and 
after the guantization values inputted from the compression frame buffer 112. 
That is, it uses several guantization values around the value inputted from the 
compression frame buffer. 

[0084] In the quantization means 111, when the quantization is conducted by 
using the different quantization value, the data whose bit amount is different can 
be produced. 

[0085] For example, when the quantization is conducted by the quantization 
value Q=2, on the input stream in which an image plane size is CIF (352 x 288) 
and the frame rate is 30 fps, and the MPEG 4 stream is mad e created , the bit rate 
is about 1.6 Mbps , and li k e a s . Similarly, at Q=6, the bit rate is about 384 kbps, 
at Q=16, about 128 kbps, and at Q=30, about 56 kbp s, the data whoso bit rat e i s 
different . Different bit rates, can be produced corresponding to tfre different 
quantization values Q. 
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[0086] The variable length coding means 113 conducts variable length coding 
on the DCT coefficient quantized by the quantization means 111, and produces 
the correction data, and produc e s the rate correction data having the number of 
areas, the number of correction data of each area, the area number and the data 
size of respective correction data as the header information, and outputs it to the 
compression frame data combination means 114. Herein, the data produced by 
the variable length coding means 113 is defined as the rate correction data. In 
FIG. 15, the structure of the rate correction data is shown , and th e . The content 
of the rate correction data is shown in FIG. 16. Furth e r. th e The structure of the 
rate correction data header is shown in FIG. 17. In FIG. 17, the rate correction 
data header 1502 has a structure in which the number in each area of the 
correction data whose bit amount is different, which is produced by changing the 
number of areas and the quantization value, and the area number and the bit 
amount of respective correction data are stored as the fixed length data. Herein, 
the ord e r of areas i s d e f i n e d as th e ordor in wh i ch tho bit amount i s l arqor are 
stored in descending order according to the bit amount . Then, in FIG. 15, the rate 
correction data has th e structur e i n which th e corr e ction data is stor e d 
succ ee d i ng to th e rato correct i on data hoador 1502 i n ordor of th e ar o a i n wh i ch 
tho b i t amount of th e corroct i on data i s l arq e r are stored in descending order 
according to the bit amount, following the rate correction data header 1502 . The 
structure of the compression frame data is shown in FIG. 13 . I n F I G. 13 . where 
the compression frame data ha sis shown with a structure in which the rate 
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correction area data and the rate correction data are successively stored after 
the normal frame data. 

[0087] The compression frame data combination means 114 stores the 
normal frame data inputted by the variable length coding means 105, the 
correction area data inputted by the reference inhibition area memory means 
109 t and the rate correction data inputted by the variable length coding means 
113 in order A as shown in FIG. 13 . and outputs th e m . The compression frame 
data combination means 114 outputs this information as the compression frame 
data 1301 to the output means 1 1 5[[, andl l. It further outputs the frame coding 
end signal showing that the coding of one frame is completed, to the frame input 
means 117. The structure of the moving picture data is shown in FIG. 14 . I n F I G. 
44, where the moving picture data 1401 bas is shown with a structure in which the 
compression frame data is successively stored. 

[0088] Thus coded moving picture data 1401 has a structure in which, for 
each frame, the normal frame data, the rate correction area data showing the 
area in which the rate correction data exists, and the rate correction data whose 
bit amount is different, are arranged in order. Then, the area having the rate 
correction data, that is, the reference inhibition area^ inhibits the reference from 
the next frame at the time of motion estimation, and because it is in the condition 
that the motion estimation is not received from the next frame, even when the 
data of this area is replaced with the rate correction data and the bit rate change 
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is conducted, the motion compensation error is not generated when the next 
frame is decoded. 

[0089] N e xt, th e structur e of th e mov i ng picture cod i ng apparatus by wh i ch 
th e mov i ng picture data coded by th e mov i ng p i ctur e cod i ng apparatus shown i n 
FIG. 1 i s us e d as th e i nput data, and th e rat e contro l is conduct e d w i thout 
d e cod i ng th e i nput data, and th e rat e chang e d mov i ng pictur e data is now l y 
produc e d, is shown in F I G. 2. The structure of the moving picture coding 
apparatus 201 is shown in Fig. 2. The moving picture coding apparatus 201 
codes the moving picture data, which it receives from the moving picture data 
producing apparatus 101. Rate control is conducted without decoding the input 
data and new rate changed moving picture data is produced. 

[0090] In FIG. 2, the moving picture coding apparatus 201 has a data 
separation means 207 connected to an input means 202, bit amount calculation 
means 203, rate correction data selection means 204, bit rate control means 205 
and moving picture combination means 208, and has a structure in which it is 
connected to an output means 206. 

[0091] The operation of thus structur o d the moving picture coding apparatus 
201 will be described below. In FIG. 2, the input means 202 inputs the 
compr e ss i on compressed and coded moving picture data 1301 and the 
ob ie ct i v e desired bit rate which is determined by the user, to the data separation 
means 207. When the data is inputted by the input means 202, the data 
separation means 207 inputs the ob ie ctiv e desired bit rate for a frame by e ach 
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on e fram e to the bit rate control means 205 , and furth e r, tak e s out th e data i n 
ord e r from th e le ad i ng data of th e inputt e d moving p i ctur e d a t a , and inputs th e 
norm al fram e data for e ach on e fram e to th e b i t amount ca l cu la t i on m e ans 203, 
and i nputs the rat e corr e ct i on ar e a data and tho rato corroct i on data for oach ono 
framo to th e rat e corr e ct i on data s e lection m e ans 20 4. The data separation 
means 207 also takes out the data in order from the leading data (first to last), 
and inputs the normal frame data for a frame to the bit amount calculation means 
203. In addition, that data separation means 207 inputs the rate correction area 
data and the rate correction data for a frame to the rate correction data selection 
means 204. When H e r ei n. wh e n the input means 202 inputs the frames other 
than the first eRe-frame of the moving picture data, after the frame coding end 
signal is received from the rate correction data selection means 204, it conducts 
the respective data input. 

[0092] The bit amount calculation means 203 calculates the bit amount of the 

inputted normal frame data[[, and]L U outputs the bit amount to the bit rate 

control means 205, and outputs the normal frame data to the rate correction data 
selection means 204 . 

[0093] Th e b i t rat e control m e ans 205 comp a r e s th e obj e ct i v e b i t rat e i nputt e d 
from the inputt e d m e ans 202 to tho current b i t amount inputted from th e b i t 
amount dotocting m e ans 203, and th e insuffic i ent b i t amount nocossary for 
satisfy i ng tho object i v e b i t rate, or suffici e nt bit amount is obtained, and is 
outputted to tho rate correct i on data s ele ction moans 204 T he bit rate control 
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means 205 compares the desired bit rate to the current bit amount inputted from 
the bit amount calculation means 203. The bit rate control means 205 obtains 
the excessive or insufficient bit amount necessary for achieving the desired bit 
rate. The excessive or insufficient bit amount necessary for achieving the 
desired bit rate is outputted to the rate correction data selection means 204. 

[0094] Th e rat e correct i on data s ele ct i on m e ans rocoives th e b i t amount 
wh i ch i s a b i t rate e rror, i nputt e d by the b i t rat e contro l moans 205, and i n ordor 
to sat i sfy th e obj e ctiv e b i t rat e , to th e ar e a shown by th e rato corroction aroa data 
i nputt e d by th e data s e parat i on m e ans 207, i t compar e s th e bit amount of th e 
ar e a i n th e normal frame data to tho b i t amount of th e p l ura li ty of corroct i on data 
stor e d i n th e rat e corroction data h e ader 1502, and wh e n th e data i s rep l aced, 
tho corr e ction data by which tho bit rato orror bocomos sma ll , i s so l octod i n ordor 
of stor e d ar e as, and by replac i ng th e data i n tho norma l fram e data w i th s ele ct e d 
corroct i on data, tho bit amount i s changed. Further, wh e n th e b i t amount e rror is 
l arg e , the corr e ction data is s olo ctod from th e n e xt aroa, and by r e p l ac i ng th e 
data, th e b i t amount i s ch a ng e d. By r e p e at i ng th e abov e proc e ssing, th e mov i ng 
p i ctur e data i n which th e bit a mount e rror i s mad e min i mum, i s outputtod to tho 
mov i ng pictur e data combin a t i on m e ans 208, and on e fram e cod i ng e nd signal i s 
outputtod to th e data s e parat i on moans 207. In order to satisfy the desired bit 
rate, the rate correction data selection means 204 receives the bit amount which 
is a bit rate error from the bit rate control means 205. For the area shown by the 
rate correction area data inputted by the data separation means 207, the rate 
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correction data selection means 204 compares the bit amount of the area in the 
normal frame data to the bit amount of the plurality of correction data stored in 
the rate correction data header 1502. When the data is replaced, the correction 
data for reducing the bit rate error is selected in order of stored areas. By 
replacing the data in the normal frame data with selected correction data, the bit 
amount is changed. Further when the bit amount error is large, the correction 
data is selected from the next area, and by replacing the data, the bit amount is 
changed. By repeating the procedure above, the moving picture data in which 
the bit amount error is reduced, is outputted to the moving picture data 
combination means 208, and one frame coding end signal is outputted to the 
data separation means 207. 

[0095] The moving picture data combination means 208 connects the frame 
data inputted for each frame from the rate correction data selection means 204 in 
order, and produces the moving picture data and outouts which is outputted to the 
output means 206. 

[0096] Herein, when the area is selected by the rate correction data selection 
means 204 and the data is replaced corresponding to the bit amount, there is a 
problem in that the motion compensation error is caused by the replacement of 
the data which is referred to as the motion compensation after l z frames in the 
conventional method, and th e resulting in image quality deterioration i s caus e d, 
howovo r . However, in the present invention, motion compensation error is not 
created, because the reference to the area having the rate correction data is 
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inhibited by the reference inhibition area memory means 109 i n th e mov i ng 
p i ctur e data produc i ng apparatus i n F I G. 1 , even when the correction data is 
selected and replaced! — the — motion — comp e nsation — error — is — net — c a us e d . 
Accordingly, the rate control can be quickly conducted by selecting the correction 
data without decoding the moving picture data and without causing the image 
quality deterioration due to the motion compensation error. 

[0097] Further, in the present embodiment, the number of areas detected by 
the maximum bit area detecting means 110 is determined by the user 
corresponding to[[, to]] what [f%11 percent the bit rate of the input data can be 
changed, and the larger the number of areas[[ is]], the larger the range of the bit 
rate change i s chanqes . However, when the number of areas is increased, 
bocauso tho roferonc e ar e a us e d for th e mot i on estimat i on b e com e s narrow, the 
coding efficiency i s l ow e r e d decreases, because the reference area used for the 
motion estimation becomes lower . In order to solve the problem, the maximum 
bit area detecting means 110 detects the area^ from the area having the 
maximum bits^ in the order i n which of the bit amount is larq e rf rom the largest . It 
is because the compr e ssion compressed and coded data does not always have a 
uniform bit amount in the frame[[,]] and there are cases where the area having 
the larger bit amount exists locally, and wh e nj f the correction data is provided te 
suGhin the area i n which th e having a large bit amoun t is larg e, the bit rate change 
becomes easv simple . and the reference inhibition area can be reduced. 
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[0098] Further, the number of rate correction data of each area selected by 
the maximum bit area detecting means 110 and the value of respective 
quantization value also contribute to the width of the bit rate change. For 
example, when the quantization of the normal frame data is conducted at Q=6, 
theits bit rate is about 384 kbps, and as th e ar e a hav i ng tho rato correct i on data, 
a p l ura li ty of ar e as hav i ng th e data amount of about 3/ 4 of th e whol e i n tota l ar e 
so l ectod from th e ar e as whos e b i t amount i s l arq e the several areas, which have 
correction data with the data amount about 3/4 of the total are selected in the 
order of bit amount from the largest . When the deviation of the bit is considered, 
an aroa size of the areas is l owo r less than 3/4. For these areas, by using the 
quantization values Q=2, Q=30, two kinds of rate correction data of about 1.6 
Mbps, and about 56 kbps are produced. Whole data size including the rate 
correction data is about 1.6 Mbps. WJre ftThe bit rate of the moving picture data 
including the rate correction data is comb i n e d w i th tho rato corroct i on data, tho 
b i t rat e chang e can b e arb i trar il y conduct e d in th e rang e of abou t variable within 
about 1 .5 Mbps to a bout 64 kbp s, and th e mov i ng p i cture data matched w i th b i t 
fates — of var i ous transmission — paths — can — be — produc e d by changing the 
combination of the rate correction data, and the moving picture data can be 
matched to the bit rates of various transmission paths . 

[0099] Further, the calculation cost relating to the rate change is low, and a 
plurality of streams i n wh i ch b i t rat e s are d i fforont. having different cit rates can be 
quickly produced. Further, when the streams whoso b i t rato i s d i ffer e nt i s are 
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prepared from th e in i tia l t i m e initially , the bit rates [[is]]are fixed and ffitsllthe data 
size becomes large. Compar i ng to that cas e , i n th e pr e s e nt e mbod i m e nt, i t is 
e nough wh e n th e data s i z e is slight l y l arg e r than th e m a x i mum bit rat e which is 
pr e sum e d Compared with such a condition, a slightly larger bit rate than the 
presumed maximum bit rate is a sufficient data size for the present embodiment . 

[0100] As described above, in the present embodiment, the motion 
compensation means A which does not conduct the motion compensation from the 
reference inhibition area, and the moving picture data producing apparatus^ 
which produces the moving picture data hav i ng tog e th e r alonq with the rate 
correction data, and the area selection means A which selects the rate correction 
data corresponding to the bit rate, are provided , thereby, . Accordingly the 
apparatus can quickly conduct th e perform rate control without decoding the 
moving picture data[[,]] and without causing [[the]] image quality deterioration due 
to [[the]] motion compensation errorr-afi d. In addition, the apparatus can quickly 
produce a plurality of streams in which the bit rate is differen t, thereby, i ts 
pract i cal eff e ct i s larg e. 

[0101] Embodiment 2 

[0102] I n th e The second embodiment^,]] will be discussed below. It is an 
apparatus i n wh i ch th e for performing rate control io conducted from tho 
provious l vw ithout decoding coded moving picture data w i thout being decod e d, 
and tho moving p i ctur e data i s n e w l y produc e d , and wh i ch i s further characterized 
in that, [[as]]for the area selection method to produce the rate correction data, the 

Substitute Specification, Marked-up copy Page 35 of 91 



Serial No.: 09/899,907 
Filed: July 5, 2001 
Conf.No.:3598 
Atty. Docket No.: 33782 

known area to which is hard l y r e f e rr e d to th e likelihood of reference at the time of 
motion estimation is low , is used , w ill b e d e scr i b e d b e low . 

[0103] In the present embodiment, the moving picture coding apparatus 201 
by wh i ch tho rato change is conducted from tho prev i ously coded mov i ng p i cture 
data, and the moving picture data i s n e w l y produc e d it produces , [[is]]are the 
same as in Embodiment 1. 

[0104] A moving picture data producing apparatus by wh i ch th e for producing 
moving picture data A which is an input [[of]]to the moving picture coding 
apparatus to qu i ck l y conduct th e rat e control, i s pr e v i ous l y produc e d , will be 
described below. 

[0105] In FIG. 3, the structure of the moving picture data producing apparatus 
by wh i ch th e mov i ng pictur e data hav i ng th e data structur e to quick l y conduct th e 
rat e contro l i s produc e d 301 , is shown. The apparatus produces moving picture 
data structured for quick performance of rate control. 

[0106] In FIG. 3, the moving picture data producing apparatus 301 is provided 
with: a frame input means 117 connected to an input means 116; motion 
compensation means 102 ; DCT conversion means 103; quantization means 104; 
variable length coding means 105; inverse quantization means 106 for 
conducting the decoding; inverse DCT conversion means 107; and frame 
memory 108 for storing the decoded frame , and the apparatus is connected to 
th e var i ab le le ngth coding m e ans 105, and i s prov i ded w i th: th e rate correction 
area selection means 310 for selecting the area producing the rate correction 
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data; reference inhibition area memory means 109; compression frame buffer 
112 to connect to the DCT conversion means 103; quantization means 111 to 
connect to the compression frame buffer and to conduct the quantization; and 
variable length coding means 113 , and i s connoctod to th e ; variable length 
coding means 1050 : and reference inhibition area memory means 109 , and 
variable l ength cod i ng moans 113, and i s provided w i th a compression framo 
data combination moans 11 4 to combin e th e mov i ng p i ctur e data, and has a 
structur e which i s conn e ct e d to an output moans 115 . The moving picture data 
producing apparatus 301 is further provided with a compression frame data 
combination means 114 to combine the moving picture data. The compression 
frame data combination means 114 is connected to both the variable length 
coding means 105. 113, the reference inhibition are memory means 109, and the 
output means 115. 

[0107] Operations of thus structured moving picture coding apparatus will be 
described below. 

[0108] In FIG. 3, operations other than the rate correction selection means 
310 are entirely the same as in Embodiment 1. In FIG. 3, the variable length 
coding means 105 conducts the variable length coding on the quantized DCT 
coefficient inputted from the quantization means 104 in the same manner as in 
Embodiment 1, and outputs it to the rate correction area selection means 310 
and the compression frame data combination means 114. Herein, the data coded 
by the variable length coding means 105 is called the normal frame data. 
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[0109] As described above, when the coding of the normal frame data is 
completed, the rate correction data area selection means 310 selects the area for 
the rate correction from the frame coded in the variable length coding means 
105, and outputs the rate correction area data 1201 show i ng indicating the 
selected area as shown in FIG. 12, to the reference inhibition area memory 
means 109 and the compression frame buffer 112. Herein, the area selected as 
the rate correction area is an area to which the probab ili ty to b e r e f e rr e d 
fro mlikelihood of reference by the next frame at the time of the motion estimation, 
is low, for example, such as a rim portion of the frame, and is defined as the 
known area which is previously stored in the rate correction data area selection 
means 310. 

[0110] In th e pr e s e nt e mbodim e nt, i t is i nhib i t e d that tho area for rat e 
corr e ct i on is r e f e rred to at th e t i m e of motion e stimat i on, and by inhibit i ng th e 
r e f e r e nc e , because the s e arch ar e a at th e t i m e of mot i on e stimat i on i s sma ll , 
thoro i s a poss i b ili ty that tho prod i ct i vo coding effic i ency is lowered. Th e r e for e , 
tho area wh i ch i s hard l y r e f e rr e d to at th e t i m e of motion e stimat i on, i s d e f i n e d i n 
such a m a nn e r th a t it i s s ele ct e d as th e rat e corr e ction ar e a. Th e r e by, e v e n wh e n 
th e s e arch ar e a at th e tim e of mot i on e stimat i on b e com e s small, b e caus e th e 
ar e a has th e sma ll probab ili ty to b e or i g i na ll y r e ferr e d to, tho practica l search 
aroa i s not sma ll , that i s, tho prod i ct i vo cod i ng efficiency can be pr e v e nt e d from 
l oworing. Operat i ons aftor tho compress i on framo buffer 112 ar e th e sam e as i n 
Embod i m e nt 1 . In the present embodiment, reference to the rate correction area 
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at the time of motion estimation is inhibited. This may reduce predictive coding 
efficiency because the search area at the time of motion estimation is small. 
Therefore, the area that is less likely to be referred to (low probability of being 
referred to) at the time of motion estimation, is selected as the rate correction 
area. Accordingly, even when the search area at the time of motion estimation 
becomes small, because the area has a low likelihood to be originally referred to. 
the practical search area is not small. In other words, it is possible to prevent a 
reduction in the predictive coding efficiency. Operations after the compression 
frame buffer 112 are the same as in Embodiment 1 . 

[0111] Thus coded moving picture data has the following structure hav i ng, 
tefor each frame[[,]]; the compression coded normal frame data, rate correction 
area data showing the area in which the rate correction data exists, and rate 
correction data including a plurality of correction data whose bit amounts [[is]]are 
different[[, and]] . Furthermore, the area having the rate correction data is i n the 
cond i t i on such that it is not referred to at the time of motion estimation from the 
next frame. 

[0112] As described above, when the coded moving picture data is an input, 
and by using the moving picture coding apparatus 201 shown in FIG. 2[[,]] in the 
same manner as in Embodiment 1, [[the]] rate correction data corresponding to 
the ob ie ct i v e desired bit rate is selected[[,]] and it i s r e p l aced w i th tho replaces 
data in the normal frame data[[, by]] . By producing the new moving picture data, 
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[[the]] rate control can bo qu i ckly conduotod w i thout d e cod i ng th e data withoyt 
decoding can be quickly conducted . 

[0113] Further, the area to produce the rate correction data is defined as the 
known area, such as the rim portion of the frame which is J=iardl vless likely to be 
referred to at the time of motion estimation, and the size of the area can be 
determined by the user corresponding to the range of the change of the bit rate. 
Further, the number of the rate correction data and the quantization value in each 
area also contribute to the width of the bit rate change. 

[0114] For example, when the quantization of the normal frame data (size: 
CIF, frame rate 30 fps) is conducted at Q=24, the bit rate becomes about 100 
kbps. [[Toll Contrarv to this, the area of th e s i z e of tota l o f in which motion 
estimation is less likely to be conducted and whose size is in total about 40% at 
the ratio of th e ar e a of the frame (are ratio) , i n which th e motion e stimat i on i s 
hard l y conduct e d, is defined as the rate correction data area , and on rospoct i vo ^ 
On said rate correction data are a data , the quantization is conducted at [[the 
]]two quantization values Q=16 and 31, and the rate correction data having the 
bit rate corresponding to about 128 kbps, and about 32 kbps are produced. With 
the moving picture data having these rate correction data, when the rate 
correction data is combined, the bit rate can be arbitrarily changed between 
about 128 kbps and about 64 kbps, and the moving picture data corresponding to 
the fluctuation of the band of the transmission path can be quickly produced. 
Further, the calculation cost according to the bit rate change is low, and a 
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plurality of moving picture data whose bit rate is different, can be quickly 
produced. 

[0115] In the present embodiment, the range of the bit rate change is smaller 
as comparod to than that of Embodiment 1— but . However in the present 
i nv e nt i on i s charact e riz e d i n tha t embodiment , because the area which is set as 
the reference inhibition area , has th e charact e r wh i ch i s hard l y for e cast, th e 
l oworing of tho coding offic i oncv can bo provontod is less likely to be predicted, 
reduction of the coding efficiency can be prevented . Further, the data size of the 
moving picture data is about 128 kbps, and almost equal to the maximum value 
of the bit rate change. 

[0116] As doscribod abovo, in tho present ombodimont, wh e n th e mov i ng 
p i ctur e data produc i ng apparatus in wh i ch a plura li ty of rat e correct i on data 
whos e b i t amount ar e d i ff e r e nt, ar e produced, to th e ar e a to which tho probabil i ty 
to b e r e f e rr e d at th e t i m e of mot i on est i mat i on i s l ow, and th e contro l m e ans 
wh i ch i s matched w i th tho objoct i vo b i t rato, and so l octs tho rato correction data, 
and conducts th e rat e contro l , ar e prov i d e d, b e caus e th e pr e d i ct i v e coding 
e ffici e ncy at th e t i m e of mot i on e st i mat i on i s not low e r e d, and th e rat e contro l can 
b e quick l y conduct e d w i thout causing th e imag e qua li ty dotor i orat i on duo to th e 
mot i on compensation orror, a p l ura li ty of moving picture data whos e b i t rat e s ar e 
d i fferent, can b e quick l y produc e d, th e r e by, i ts pract i ca l e ff e ct is l arg e . As 
described above, the present embodiment provides a motion picture data 
producing apparatus, which creates several rate correction data with a different 
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bit amount for the area less likely to be referred to at the time of motion 
estimation. This embodiment also provides a rate controlling means, which 
selects rate correction data and controls the rate corresponding to the desired bit 
rate. These elements make possible high-speed rate controlling without either 
the reduction of predictive coding efficiency at the time of motion estimation or 
image gualitv degradation caused by motion compensation error, and the high 
speed creation of several motion picture data with different bit rates. 
[0117] Embodiment 3 

[0118] In the third embodiment, described below, the moving picture data 
producing apparatus ha s i n wh i ch th e r a t e contro l i s conduct e d from th e 
previous l y codod mov i ng p i cture data w i thout b e ing d e cod e d, and the mov i ng 
pictur e data i s n e w l y produc e d, wh e r e in th e apparatus has, to th e mov i ng pictur e 
data, by using the referred degree at the time of the motion estimation, a means 
for selecting the rate correction area to produce a plurality of data in which the bit 
amount is different , w il l bo doscr i bod bo l ow . 

[0119] Further, in the present embodiment, the moving picture coding 
apparatus in wh i ch th e r a t e contro l is conduct e d from th e pr e v i ously cod e d 
mov i ng p i ctur e dat a , and tho mov i ng p i cture data is newly produc e d, is ent i re l y 
the same as in Embodiment 1. 

[0120] The structure of the moving picture data producing apparatus is shown 
in FIG. 4. In FIG. 4, a moving picture data producing apparatus 401 has: a frame 
input means 117 connected to an input means 116; motion compensation means 
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402; DCT conversion means 103; quantization means 104; variable length 
coding means [[10511 405 ; inverse quantization means [[10511 106 for decoding; 
inverse DCT conversion means 107; [[and]] frame memory 108 for storing the 
decoded frame[[, and]]; a referred area memory means 410 connected to a 
motion compensation means 402; and a rate correction area selection means 
412[[, and]]; a compression frame buffer 112 connected to a DCT conversion 
means [[404H 103 ; quantization means 111; and variable length coding means 
113 , and a rat e correct i on ar e a s el ect i on moans 412 ; compression frame 
combination means 414 connect e d to th e var i abl e l e ngth coding m e ans 113 and 
var i abl e le ngth cod i ng m e ans 105, and has a structure i n wh i ch thoso moans aro 
conn e ct e d to th e ; and an output means 115. 

[0121] Operations of thus structured moving picture producing apparatus will 
be described below. In the moving picture coding apparatus 401, the input 
means 116, frame input means 117, DCT conversion means 103, quantization 
means 104, variable length coding means 105, inverse quantization means 106, 
inverse DCT conversion means 107, and frame memory 108 produce the normal 
frame data in e nt i r el y the same manner as in Embodiment 1. In this connection, 
the variable length coding means 405 outputs the frame coding end signal to the 
frame input means 117 when the normal frame data produc i ng is comp l ot e d has 
been produced . In this manner, when the normal frame data for one frame is 
produced, the frame input means 117 outputs the next non-compression frame to 
the motion compensation means 402. 
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[0122] Th e motion comp e nsation m e ans 4 02 do e s not conduct th e mot i on 
comp e nsat i on on th e I fram e , and outputs to th e DCT conv e rs i on m e ans 103, 
and i n th e cas e of oth e r than I fram e , th e mot i on comp e nsat i on i s conduct e d by 
us i ng tho procod i ng framo by I framo and tho framo i nputted by tho framo i nput 
m e ans 117. Furth e r, i n th e pr e ceding fram e by 1 fram e , the area i nformat i on 
wh i ch i s ref e rr e d to at the t i m e of mot i on e st i mation, i s outputt e d to th e r e ferred 
aroa memory moans 4 10. The motion compensation means 402. which outputs to 
the DCT conversion means 103, does not conduct the motion compensation on 
the l-frame. In the case of other than l-frames, the motion compensation is 
conducted by using the immediately preceding frame and the frame inputted by 
the frame input means 117. Further, in the immediately preceding frame, the 
area information which is referred to at the time of motion estimation is outputted 
to the referred area memory means 410. 

[0123] The referred area data is shown in FIG. 18. The referred area data is 
the data in which the referred degree of each area is stored. I n tho ref e rr e d ar e a 
data. tho The referred degree of each area m ea ns th a t, i n p i x el s i n th e ar e a, tota l 
p i x el numb e r wh i ch i s is the total number of pixels referred to at the time of motion 
estimation from the next frame , is shown, a nd in F I G. 18, i t is d e fin e d th a t th e 
th i ck e r the co l or of tho aroa i s, tho h i gh e r th e r e f e rr e d d e gr ee i s . In Fig. 18. the 
dark shading indicates a more highly referred degree. 
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[0124] The referred area memory means 410 stores the referred area inputted 
from the motion compensation means 403, and outputs the referred area data 
showing the referred area to the rate correction area selection means 412. 

[0125] The rate correction area selection means 412 selects, in the inputted 
referred area data, a predetermined number of areas in the order from the area 
in which the referred area is low, as the rate correction data area. Further, the 
selected area is the rate correction area in the preceding frame immediately 
preceding the by on e fram o to the fram e currently inputted by the frame input 

means[[, and]], It outputs the rate correction area data 1201^ showing the 

selected rate correction area^ to the compression frame buffer 112 and the 
moving picture data combination means 414. Furth e r, th e The operation of the 
compression frame buffer 112, quantization means 111, and variable length 
coding means 113 is the same as in Embodiment 1. 

[0126] The compression frame combination means 414 combines the normal 
frame data inputted from the variable coding means 405, the rate correction area 
data inputted from the rate correction area selection means 412, and the rate 
correction data inputted from the variable length coding means 113 as shown in 
FIG. 13, and outputs it to the output means 115. In the present embodiment, 
because i t is not conduct e d that the reference inhibition area is not set at the time 
of motion estimation and the search area is limited, as shown in Embodiments 1 
and 2, the search area is not limited, and the predictive coding effect can be 
more enhanced than in Embodiments 1 and 2. 
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[0127] By using the coded moving picture data as an input as described 
above, and the moving picture coding apparatus in FIG. 2, when the new moving 
picture data is produced in the same manner as in Embodiment 1, the rate 
control can be quickly conducted without decoding the data. Further, becaus e, as 
the area having the rate correction data, the area to which the referred degree is 
low at the time of motion estimation, is selected, even when the rate correction 
data is selected at the time of the rate control, the motion compensation error is 
not caused created . Further, the number of area to be selected in the order from 
the area in which the referred degree is lower, can be determined by the user 
corresponding to the range of the bit rate change. Further, the number of rate 
correction data and the quantization value in respective areas also contribute to 
the width of the bit rate change. 

[0128] For example, when the quantization of the normal frame data (size: 
GIF, frame rate 30 fps) is conducted at Q=24, the bit rate is about 100 kbps. In 
contrast to th i s , the area [[in ]]which th e r e f e rr e d d e gr ee i s l ow and th e has a 
likelihood of being referred to and whose size is the total [[of]] about 30% i n th e 
ar e a ratio of the frame (area ratio) , is defined as the rate correction data area, 
and to respective areas, the quantization is conducted at two quantization values 
of Q=16 and 32[[, and thell . The rate correction data having the bit rate 
corresponding to about 128 kbps and about 32 kbps is produced. With the 
moving picture data having this rate correction data, when the rate correction 
data is combined, the bit rate can be arbitrarily changed between about 128 and 
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about 64 kbps[[.]] . Accordingly, the moving picture data corresponding to the 
fluctuation of the band of the transmission path can be quickly produced. Further, 
the calculation cost according to the bit rate change is low, and a plurality of 
moving picture data whose bit rate are different, can be quickly produced. 
Further, the data size of the moving picture data is about 128 kbps, and it is 
almost equal to the maximum value of the bit rate change. In this example, the 
range of the bit rate change is smaller than in Embodiment 1[[, however]].. 
However , because the reference inhibition area is not provided, the lowering of 
the coding efficiency can be prevented, and because the area having the rate 
correction data is an area to which the practical referred degree is low, the 
generation of the motion compensation error which accompanies the 
replacement of the data can be suppressed to sma ll minimized . 

[0129] As described above, [[in]] the present embodiment^, the]] provides a 
moving picture data structure having the rate correction data [[to]]for the area 
[[to]]in which the referred degree is low at the time of motion estimation , and th e,. 
Further provided is a moving picture data producing apparatus which selects the 
rate correction data and changes the bit rate , ar e prov i d e d, and th e r e by^ 
Accordingly , because the predictive coding efficiency of the motion estimation is 
not l ow e r e d reduced, and the motion compensation error generated when the rate 
correction data is used is lowered, and the rate control can be quickly conducted, 
a plurality of moving picture data whose bit rates are different, can be quickly 
produced , and its pract i ca l eff e ct i s l arg e. 
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[0130] Embodiment 4 

[0131] Ifi-th eThe fourth embodiment^, inll is an apparatus [[by]]jn which [[the]] 
previously coded moving picture data is not decoded, and the rate control is 
conducted, and the new moving picture data is produced , tho apparatus 
charact e r i zod in tha t . In this embodiment, discussed below, the input moving 
picture data has the rate correction data whose bit amount i s with slightly different 
bit amounts . [[on]]for the entire fram o, w ill bo doscr i bod bolow . 

[0132] In F I G. 5. th eFiq. 5 shows the structure of the moving p i ctur e d at 
apicture data producing apparatus 501 [[by]]in which the-coding is pr e v i ous l y 
conducted for the input data [[of]]to the moving picture coding apparatus-te 
conduct tho rato contro l , i s shown . The moving picture coding apparatus 
performs rate control on the input data. In FIG. 5, the moving data producing 
apparatus 501 is provided with: the frame input means 117 connected to the 
input means 116; motion compensation means 102; DCT conversion means 103; 
quantization means 104; and-variable length coding means 105 , and is prov i ded 
w i th th e ; inverse quantization means 106 for conducting the decoding; inverse 
DCT conversion means 107; and frame memory 108. Further, the moving picture 
coding apparatus 501 is provided with: a low pass filter 503 connected to the 
frame input means 117; motion compensation means 102 connected to the low 
pass filter 503; DCT conversion means 103; quantization means 104; and 
variable length coding means 106, and i s prov i ded w i th: tho 105; inverse 
quantization means 106 which is connected to the quantization means 104 and 
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conducts the decoding; inverse DCT conversion means 107; and frame memory 
108. Furthe r, th e mov i ng p i ctur e data producing app a ratus 501 i s structur e d i n 
such a manner that it i s conn e ct e d to th e variabl e le ngth cod i ng m e ans 105, th e 
compross i on framo data comb i nat i on moans 504 conn e ct e d to the var i ab le 
l ength coding moans 105. and output means 115 included are a compression 
frame data combination means 504 connected to the output means 115 and both 
variable length coding means 105 . 

[0133] The operation of thus structured moving picture data producing 
apparatus will be described below. In the moving picture coding apparatus 501, 
tehthe frame input means 117, motion compensation means 102, DCT 
conversion means 103, quantization means 104, variable length coding means 
105, inverse quantization means 106, inves rin verse DCT conversion means 107, 
and frame memory 108 are blocks for producing the normal frame data in the 
same manner as in Embodiment 1. 

[0134] Further, in FIG. 5, the low pass filter 503 causes the f*ob- 
compr e ssion non-compressed frame to pass through the low pass filte r 503 , 
reduces the high frequency information of the input frame, and outputs it to the 
motion compensation means 102. Hereinafter, by using the DCT conversion 
means 103, quantization means 104, variable length coding means [[506]]105, 
inverse quantization means 106, inverse DCT conversion means 107, and frame 
memory 108, in the same manner as in the normal frame data production, by 
using the same quantization value, the frame data is produced. In this 
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connection, the frame data produc e d h e r e in i s th e data produc e d i n such a 
mann e r that tho data i n wh i ch the h i gh fr e qu e ncy of th e i nput fram e i s r e duc e d 
b vthat passes through the low pass filter 503[[,]] is coded, and is the data in 
which its bit amount is f e w e r less than the normal frame dat a, and has th e 
structur e i n wh i ch i t has th e corr e ct i on data t o . Furthermore, this data is 
structured to contain the correction data for the whole area m- Aof one frame 
shown in FIG. 10, and this data is called the rate correction data. The variable 
length coding means [[50611 105 calculates[[, to]] the rate correction data[[,]] 
header for the rate correction data h e ad e r as shown in FIG. 20, and outputs the 
rate correction data and the rate correction data header to the moving pictur e 
comb i nation compression frame data combination means 20. The rate correction 
data header has the structure having the number of areas in the rate correction 
data fnil for one frame[[,]] and the bit amount in each area. 

[0135] The compression frame data combination means 504 combines the 
normal frame data inputted from the variable length coding means 105, rate 
correction data header inputted from the variable length coding means 506, and 
rate correction data as shown in FIG. 19, and outputs it to the output means 115. 

[0136] As d e scr i b e d abov e , i n th e pr e s e nt e mbodim e nt, th e structur e hav i ng 
tho rato corr e ct i on data i s appl ie d to a ll aroas of 1 frame, and respect i ve data 
amounts of the norma l frame outputted from th e variab l e length cod i ng m e ans 
105, and the rate correct i on data outputtod from tho var i ab le le ngth cod i ng 
m e ans 106, ar e s li ght l y different, by reduc i ng tho high fr e qu e ncy compon e nt of 
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tho non compr e ss i on fram e . As described above, in the present embodiment, rate 
correction data is structured to have the rate correction data for all areas of one 
frame. The rate correction data is outputted from the variable length coding 
means 105. Corresponding normal frame data is outputted from the variable 
length coding means 105. The rate correction data and normal frame data are 
slightly different due to the reduced high freguencv component of the non- 
compressed frame. 

[0137] The structure of the moving picture coding apparatus 2101, in which 
[[the]]new moving picture data cod e d as d e scr i b e d abov e is an i nput, and th e 
rate control i s conduct e d w i thout decod i ng the data, and th e moving p i ctur e d a ta 
i s n e w l v is produced, is shown in FIG. 21. Moving picture data coded as 
described above is an input to the moving picture coding apparatus 2102. It 
performs rate control without decoding the data. In FIG. 21, the moving picture 
coding apparatus 2101 is provided with: the data separation means 207 
connected to the input means 202; bit amount calculation means 203; rate 
correction data selection means 2104; bit rate control means 205; and-moving 
picture data combination means 208 , and th e apparatus has th e structur e in 
wh i ch th e s e ar e conn e ct e d to th e ; and an output means 206. 

[0138] The op e rat i on of thus structured mov i ng p i cture coding apparatus 2101 
w ill b e d e scr i b e d b el ow. I n F I G. 21, th e op e rat i ons of b l ocks oth e r than th e rat e 
corr e ct i on data s el ection m e ans 210 4 ar e e ntire l y the samo as i n Embod i ment 1. 
Th e rate correct i on data s ele ct i on moans 210 4 conducts, by us i ng tho b i t rate 
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e rror i nputt e d from th e b i t rat e contro l m e ans 205, rat e corr e ct i on data h e ad e r 
i nputt e d from th e data s e p a rat i on m e ans 207, rat e corr e ction data, a nd norma l 
fram e data i nputt e d from th e b i t amount ca l cu l at i on m e ans 203, th e rat e contro l 
so that tho b i t rato orror i s roducod. A flow of tho process i ng of tho rato contro l i s 
shown i n FIG. 22 T he operation of the moving picture coding apparatus 2101 will 
be described below. In FIG. 21, the operations of blocks other than the rate 
correction data selection means 2104 are the same as in Embodiment 1. The 
rate correction data selection means 2104 performs the rate control which 
reduces the bit rate error. It does this by using the bit rate error inputted from the 
bit rate control means 205, rate correction data header inputted from the data 
separation means 207, rate correction data, and normal frame data inputted from 
the bit amount calculation means 203. A flow chart of the processing of the rate 
control is shown in FIG. 22. 

[0139] As shown in FIG. 22[[,]]: 

[0140] STEP 1: in i tia ll y, tho pos i t i ve or nogativo of tho bit rato orror is judg e d, 
and wh e n it i s n e g a t i v e or z e ro, th e s e qu e nc e i s compl e t e d, and wh e n i t is 
pos i t i v e , that is, b i t i s excoss i v e , Determine whether the bit rate error is positive or 
negative. When it is negative or zero, the seguence is completed. When it is 
positive, the bit rate error is excessive. 

[0141] STEP 2: [[the]]The rate correction data header is referred to, and the 
maximum area of the bit amount is selected. 
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[0142] STEP 3: to th e s ele ct e d ar e a, th e norma l fram e data is r e p la c e d w i th 
tho rate corroction data, Replace the normal frame data with the rate correction 
data for the selected area. 

[0143] STEP 4: afte fAfter the data is replaced, the bit rate error is renewed, 
and the sequence advanc e s proceeds to the bit rate error judgement process 
(STEP 1) . 

[0144] The above processing is repeated until the bit rate error becomes 
negative or zero , and when the processing is completed, the frame data is 
outputted to the moving picture data combination means 208 . The moving picture 
data combination means 208 connects the frame data inputted for each 
[[1]] sinqle frame in order, and the moving picture data is produced, and outputted 
to the output means 206. 

[0145] In the present embodiment, after the rate correction data passes 
through the low pass filter, because it is quantized at the same quantization value 
as the normal frame, there is only a tittie small difference in the bit amount, as 
compared to the normal data which is coded without_a low pass filter, and by 
selecting the rate correction data to a plurality of areas, [[the]] fine rate control 
can be conducted. 

[0146] The frequency characteristic of this low pass filter can be determined 
by the user corresponding to the range of the bit rate change. For example, in th e 
cas e wh e r e when the low pass filter hav i ng th e has a frequency characteristic in 
whte frsuch that the coding data after [[the]] passage of tho low pass throuqh the 
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filter is 64 kbps, and the filter is used to th efor moving picture data in which the 
bit rate after the coding is 128 kbps, when the input data is 128 kbps, [[the]] 
moving picture data i n wh i ch th e f i n e b i t rat e chang e i s poss i b le i n th e rang e 
b e tw ee n 128 kbps and about 6 4 kbps, can be produced in which fine bit rate 
change is possible in the range between 128kbps and about 64 kbps . Further, 
the calculation cost which accompanies the bit rate change is low, and 
thorob v therefore . a plurality of moving picture data whose bit rates are different, 
can be quickly produced. 

[0147] Further, because the difference of the image quality in the rate 
correction data and the normal data are small as compared to the case where 
the image data is produced by changing the quantization value as in 
Embodiments 1, 2 and 3, the present embodiment is characterized in that the 
motion compensation error caused by selecting the rate correction data is small. 

[0148] As d e scr i b e d abov e , in th e prosont ombod i mont, to tho who l o aroas i n 
tho framo, tho moving p i cture data structur e having togothor th e rat e corr e ct i on 
dat a wh i ch is produc e d by r e moving th e high fr e qu e ncy and b ei ng cod e d, and 
th e moving pictur e cod i ng appar a tus by which th e rat e corr e ct i on d a ta i s so lo ctod 
corr e sponding to the obj e ctive b i t rato, and tho rato contro l i s conduct e d, ar e 
prov i d e d, th e r e by, becaus e , wh i l e th e mot i on comp e nsat i on e rror is r e duc e d, th e 
fin e rat e contro l can b e qu i ck l y conduct e d, a plurality of mov i ng p i ctur e data 
whos e bit rat e s aro d i fferent, can b e qu i ck l y produced, and i ts pract i ca l e ff e ct i s 
lareev As described above, the present embodiment provides, for the whole areas 

Substitute Specification, Marked-up copy Page 54 of 91 



Serial No.: 09/899,907 
Filed: July 5, 2001 
Conf.No.:3598 
Atty. Docket No.: 33782 

in the frame, a moving picture data structure which includes the rate correction 
data. The rate correction data is created by removing the high frequency 
component and coding. The present embodiment further includes the moving 
picture coding apparatus which conducts rate control by selecting the rate 
correction data according to the desired bit rate. In this embodiment, motion 
compensation error is reduced, fine rate control is conducted, and a plurality of 
moving picture data whose bit rates are different can be quickly produced. 
[0149] Embodiment 5 

[0150] I n th e fifth e mbod i m e nt, in th e apparatus by which th e rat e control i s 
conduct e d from th e pr e v i ously cod e d mov i ng p i ctur e data, w i thout decoding tho 
dat a , — a«d — the — mov i ng — p i ctur e — data — is — n e w l y — produced, — the — apparatus 
charact e r i zed i n that the input moving p i ctur e data has tho data structure i n wh i ch 
th e b i t r e duction i s poss i b le at th e t i m e of rat e contro l , w ill bo d e scrib e d b el ow. 
Th e pr e sent apparatus is structured by 2 apparatus of tho mov i ng picture data 
produc i ng apparatus i n which th e mov i ng p i ctur e data i s pr e v i ous l y coded, and 
th e moving pictur e cod i ng apparatus in wh i ch th e rat e control i s conduct e d, and 
th e mov i ng pictur e data i s produc e d. The fifth embodiment is described below. It 
performs rate control on previously coded moving picture data without decoding 
the data, and produces new moving picture data. The apparatus is characterized 
in that the input moving picture data is structured such that bit reduction is 
possible at the time of rate control. The present apparatus is structured by two 
apparatuses, which are the moving picture data producing apparatus in which the 
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moving picture data is previously coded, and the moving picture coding 
apparatus in which the rate control is conducted and the moving picture data is 
produced. 

[0151] I nit i al l y, tho mov i ng p i ctur e data producing apparatus by wh i ch th e 
mov i ng p i ctur e data hav i ng th e structur e i n wh i ch the b i t reduct i on is poss i b l e at 
tho tim e of rato contro l , is produc e d, wi ll b e d e scrib e d, and next, th e mov i ng 
p i ctur e cod i ng apparatus by wh i ch th e rat e contro l is conduct e d and th e mov i ng 
p i cturo data i s produc e d, w ill b e d e scr i b e d. Initially described below is the moving 
picture data producing apparatus for producing moving picture data structured for 
bit reduction at the time of rate control. Next, the moving picture coding 
apparatus for performing rate control and producing moving picture data will be 
described. 

[0152] In FIG. 6, the structure of the moving picture data producing apparatus 
601 is shown. I n F I G. 6, a The moving picture data producing apparatus 601 is 
provid e d w i th includes : a moving picture coding means 603 connected to an input 
means 602; a data d i v i d i ng division position selection means 605 for selecting the 
dividable position in the data of the V i doo Packo t video packet structured by a 
continuous arbitrary number of macro blocks; video Pack e t packet termination 
data producing means 606; and rate correction data producing means 607 for 
storing the information for the rate correction , and is prov i ded w i th moving p i ctur e 
data comb i nat i on m e ans 608 conn e ct e d to th e V i doo Packet producing m e ans 
604 ; compression frame data combination means 608 connected to the moving 

Substitute Specification, Marked-up copy Page 56 of 91 



Serial No.: 09/899,907 
Filed: July 5, 2001 
Conf.No.: 3598 
Atty. Docket No.: 33782 

picture coding means 603 and the rate correction data producing means 607 T 
and has tho structure in wh i ch th e s e ar e conn e ct e d t o ; and an output means 609. 

[0153] The operation of the moving picture data producing apparatus 601 
structurod as abovo w ill b ejs described below. Initially, the input means 602 
inputs the non - compression non-compressed image into the moving picture 
coding means 603 for each [[1]] frame. Then, the moving picture coding means 
603 conducts the motion compensation, DCT conversion, quantization, and 
variable length coding processing on the inputted frame in the V i doo Packo t video 
packet unit structured by the continuous arbitrary macro block. Further, the 
moving picture coding means 603 stores respective start positions of the final 
macro block for each V i d o o Pack e t video packet at the time of the coding, and 
produces the Vid e o Pack e t video packet structure data. The V i doo Pack e t video 
packet structure data is shown in FIG. 36. In FIG. 36, the Vid e o Pack e t video 
packet structure data records the total number of the Vid e o Pack e t video packet , 
and the start position of the final macro block in respective Vid e o Pack e ts video 
packets . 

[0154] Then, the moving picture coding means 603 outputs the coded V i d e o 
Paeket video packet to the data dividing position selection means 605 and the 
movinq compression frame data combination means 608, and outputs the V i d e o 
Paeket video packet structure data to the data d i v i d i nq division position selection 
means 605. Herein, the V i doo Pack e t video packet may have the same structure 
as in the area shown in, for example, FIG. 10, but, it is necessary that each 
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V i d e o Pack e t video packet is structured by the continuous macro blocks in the 
lateral direction. 

[0155] The data d i v i d i nq division position selection means 605 selects the area 
which can be deleted at the time of rate control for the respective final macro 
blocks of the inputted V i d e o Pack e t video packet , and determines its border as 
the dividing position of the Vid e o Pack e t video packet . In each V i deo Packe t video 
packet , the continuous macro block is accommodated in orde r, and in^ Jn each 
macro block, b e caus e the variable length sie ftcode of the quantized DCT 
coefficient is accommodated in order from the low frequency side . Accordingly , 
in the final macro block of the V i doo Pack e t video packet the variable length 
sigftcode of the rear side corresponds to the high frequency DCT coefficient-ef 
th e high frequency . Accordingly, as shown in FIG. 23, the final macro block of the 
Vidoo Packo t video packet is divided at the time of rate control, and the position 
to delete the subsequent data is selected from the rear side of the final macro 
block of each Vid e o Pack e t video packet . That is, the data div i ding division 
position selection means 605 causes the high freguencv DCT coefficient of th e 
high fr e qu e ncy accommodat e d i n located at the rear side of the final macro block 
of the Vid e o P a ck e t video packet to be the area which can be deleted. Further, 
because the high frequency component is sma lle r, as compar e d to has a smaller 
influence on the image guality than the low frequency component, i n tho 
influ e nc e onto th e i mag e qua li ty even when the information is deleted, the 
deterioration of the image quality by deleting the high frequency component is 
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small. Then, the data d i v i ding division position selection means 605 outputs the 
information of the selected position[[,]] and [fitsl lthe bit amount and th e 
subsequent to the selected position and the bit amount subsequent to the 
selected position to the rate correction data producing means 607, and outputs 
the information of the selected position and each V i d e o Pack e t video packet to 
the V i deo Pack e t video packet termination data producing means 606. 

[0156] Tho V i doo Packet term i nation data produc i ng m e ans 606 ca l cu l ates, to 
th e f i rst var i ab le le ngth s i gn b e g i nn i ng from th e pos i t i on se l ect e d by th e data 
d i v i d i ng pos i tion se le ct i on m e ans 605, th e v a r i ab le le ngth s i gn wh e n th e s i gn is 
th e la st of tho V i d o o Packet, and further, as shown i n F I G. 2 4 , produc e s th e 
V i d o o Pack e t t e rm i nat i on data to which th e stuff i ng b i t for byt e a l ignm e nt wh e n i t 
is th e last of th e Vidoo Packet, i s add e d, and outputs i t to the rat e corr e ct i on data 
produc i ng m e ans 607. Horo i n, tho V i doo Packet term i nation data i tso l f is a l so 
added by th e stuffing b i t for adjust i ng th e byt e a li gnm e nt. Thon, tho rato 
corr e ct i on data produc i ng m o ans 607 col l ects th e d i v i d e d pos i t i on inputted by th e 
dat a d i v i d i ng pos i tion so l oct i on moans 605, th e bit amounts subsequent to th e 
pos i t i on, tho Vid e o Pack e t t e rminat i on data inputt e d by th e V i deo Pack e t 
t e rm i nat i on data producing m e ans 606, as th e rat e corr e ction data, and output i t 
to th e compress i on framo data comb i nat i on moans 608. FIG. 25 i s a v i ow 
showing tho rato correct i on data structur e and rat e corr e ct i on data h e ad e r 
structur e in th e pr e sent f i fth e mbod i ment. Furth e r, F I G. 26 is a viow show i ng the 
data content of tho rato corr e ct i on data shown i n FIG. 25. Tho rato correct i on 

Substitute Specification, Marked-up copy Page 59 of 91 



Serial No.: 09/899,907 
Filed: July 5, 2001 
Conf.No.: 3598 
Atty. Docket No.: 33782 

data has the structur e , as shown in F I G. 25 and F I G. 26, cons i st i ng of th e rat e 
corr e ct i on data h e ader, and a plura li ty of V i d e o Pack e t t e rm i nat i on data, and th e 
rat e corr e ction data hoador has the structur e i nc l ud i ng tho Vid e o Pack e t numb e r 
of 1 framo, b i t numb e r show i ng th e d i vided position of oach V i d e o packet, bit 
amounts which can b e d e l e t e d, subs e qu e nt to th e d i v i ded pos i tion, and b i t 
amount of th e V i d e o Packet termination data. The video packet termination data / 
producing means 606 calculates a variable length code for a first variable length 
code starting from a position selected by the data division position selection 
means 605 when the code is the last of the video packet. Further as shown in 
FIG. 24, the video packet termination data producing means 606 produces the 
video packet termination data to which the stuffing bit for byte alignment when it 
is the last of the video packet, is added, and outputs it to the rate correction data 
producing means 607. Herein, the stuffing bit is also added to the video packet 
termination data for adjusting the byte alignment. Then, the rate correction data 
producing means 607 collects the following as the rate correction data: the 
divided position inputted by the data division position selection means 605, the 
bit amounts subseguent to the position, and the video packet termination data 
inputted by the video packet termination data producing means 606. The rate 
correction data is then outputted to the compression frame data combination 
means 608. FIG. 25 is a view showing the rate correction data structure and rate 
correction data header structure in the present fifth embodiment. Further, FIG. 
26 is a view showing the data content of the rate correction data shown in FIG. 
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25. The rate correction data is comprised of the rate correction data header, and 
a plurality of video packet termination data. The rate correction data header 
includes the video packet number of a frame, bit number showing the divided 
position of each video packet, bit amounts subsequent to the divided position 
which can be deleted, and bit amount of the video packet termination data. 

[0157] Finally, the mov i ng pictur e data compression frame data combination 
means 608 combines the normal V i doo Pack e t video packet inputted by the Vid e o 
Packet produc i ng moans 60 4 moving picture coding means 603 for each [[1]] 
frame, and the rate correction data inputted by the rate correction data producing 
means 607 for each [[1]] frame, in order, and produces the moving picture data, 
and outputs it to the output means 609. 

[0158] Next, the moving picture coding apparatus will be described. The 
moving picture coding apparatusw Weh conducts the rate control and produces 
the moving picture data . Its input is from th e moving picture data produced by 
the moving picture data producing moans, wi ll bo doscribod apparatus 601 . 

[0159] The structure of the moving picture coding apparatus 701 is shown in 
FIG. 7. I n F I G. 7. the mov i ng pictur e produc i ng apparatus 701 has lt includes a 
rate correction data extraction means 703 connected to an input means 702[[,]]; 
data composition means 704[[, and]]; rate correction means 706[[, and]]; a bit 
rate control means 705 conn e ct e d to th e i nput moans 702, rat e corr e ction data 
e xtract i on m e ans 703, and data composit i on m e ans 704, and has th e structur e 
connoctod to ; and an output means 707. Tho operat i on of tho structured as 
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d e scr i b e d abov e , w i l l b e d e scr i b e d b el ow. Th e moving p i ctur e produc i ng 
apparatus 701 has th e mov i ng p i cture data outputted by tho output m ea ns 609 
as an i nput, and conducts th e rat e ch a ng e and n e w l y produc e s th e mov i ng 
pictur e data. The operation of the moving picture coding apparatus 701 is 
described below. The input to the moving picture coding apparatus 701 is the 
output from the moving picture data producing apparatus 601 provided at the 
output means 609. The moving picture coding apparatus 701 performs the rate 
change and produces new moving picture data. 

[0160] The input means 702 inputs the coded moving picture data^ including 
the V i doo Packot video packet for each [[1]] frame A to the rate correction data 
extraction means 703 , and the ob j ect i ve . It also inputs the desired bit rate into 
the bit rate control means 705. The rate correction data extraction means 703 
extracts the rate correction data from the inputted moving picture data[[,]]_and 
outputs it to the bit rate control means 706, and 705. The rate correction data 
extraction means 703 also outputs the normal V i d e o P a ck e t oth e r than th e rat e 
corr e ct i on data to video packet (excluding the rate correction data) to the data 
composition means 704. The data composition means 704 combines the inputted 
V i d o o Pack o ts video packets and composes the frame data, [[and]] outputs the 
generated bit amount to the bit rate control means 705, and outputs the 
composite frame data to the rate correction means 706. Furth e r, th e The bit rate 
control means 705 compares the obi e ct i ve desired bit rate inputted from the input 
means 702 to the generated bit amount inputted from the data composition 
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means 704, and calculates the bit rate error[[, and tol l. To satisfy the 
ob ie ct i v e desired bit rate, when the bit amount is excessive, the bit rate control 
means 705 refers to the rate correction data header in the rate correction data 
inputted from the rate correction data extraction means 703, and selects from 
which position V i d e o Pack e t th e bit i s video packets the bits are deleted[[, and]L 
The bit rate control means 705 outputs the correction data corresponding to the 
rate correction data header showing the position, to the rate correction means 
706. Herein, the selection method of the Video Pack e t video packet to delete the 
bits, is as follows: the V i d e o P a ck e t video packet is selected in th e ord e r of th e 
V i doo Pack o t i n wh i ch the b i t amount to b e d e l e t e d i n th e rat e corr e ction d a t a 
h e ad e r is l arg e r, and th e bit i s de l eted, and th e b i t amount which can b e d ele t e d, 
is th e d i fferent i at e d from th e b i t rat e error, and by add i ng th e b i t amount of th e 
t e rminat i on data, tho b i t rato orror i s ronowod. and tho descendinq order 
according to the amount of bits to be deleted in the rate correction data header, 
the bits are deleted, the bit amount which can be deleted is subtracted from the 
bit rate error, and to this is added bit amount of the termination data, which 
updates the bit rate error. The selection processing is continued until the 
objective bit rate is satisfied. 

[0161] The rate correction means 706 deletes the bits subsequent to the bit 
position which can be deleted, accommodated in the rate correction data header, 
for the Vid e o Pack e t video packet inputted by the bit rate control means 705 , and 
th e. The Vidoo Packo t video packet termination data inputted by the rate control 
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means 706 i s i ns e rt e d instead of them, and tho replaces the deleted bits. The 
frame data which is rate corrected, is outputted to the output means 707. The 
output means 707 compos e s synthesizes the moving picture data^ inputted for 
oach 1 fram e from the rate correction means 706, for each frame and produces 
the moving picture data. 

[0162] As described above, to d ele t e deleting the rear portion bits of the final 
macro block of the Vid e o Pack e t video packet so that rate correction is 
conducted[[,]] is equivalent to tba tremoving the high frequency component of the 
final macro block of the Vid e o Pack e t video packet, is r e mov e d, and h a s th e 
sam e e ff e ct a s a cas e that th e var i ab le le ngth d e cod i ng and re - quant i z a t i on ar e 
conduct e d Accordingly, this process has the same effect as the process in which 
the variable length decoding and re-quantization are conducted , and the bit rate 
is lowered. Furth e r, b e caus e Because the decoding processing is not necessary, 
the processing is light and the rate control can be quickly conducted. 

[0163] Further, the number of the rate correction data is equal to the number 
of the Vid e o P a ck e t video packet , and the number of the V i d e o Pack e t video 
packet can be set by the user corresponding to the range of the bit rate change. 
For example, in the case where the size of [[1]]one V i d e o P a cke t video packet is 
small, for example, several [[%1] percent of the whole frame, wh i c h i s sma ll, and 
the data amount of the rear portion bit which can be deleted, corresponds to 
about 10% [[to]]of the data amount of 1 V i deo Pack e t one video packet , when the 
input data is about 64 kbps, the-data in which the bit rate can be changed in the 
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range of about 64 kbps to about 56 kbps, can be produced. Thereby, in order to 
cope with the fluctuation of the network, the bit rate change can be quickly 
conducted. Further, the present embodiment is characterized in that the rate 
correction data may be smaller, as compared to Embodiments 1 to 4. 

[0164] As described above, [[in]] the present embodiment^.] ] provides: the 
V i d e o Pack e t video packet structure whose rear portion bit can be deleted, the 
moving picture data structure having the rate correction data in which the position 
information and termination data are stored, and the moving picture coding 
apparatus which refers to the rate correction data and deletes the bits of rear 
portion of the Vid e o Pack e t video packet and conducts the rate control , ar e 
prov i d e d, th e r e b y . Accordingly , the rate control can be quickly conducted without 
decoding the moving picture data , and its pract i ca l e ff e ct i s l arg e. 

[0165] Embodiment 6 

[0166] In tho s i xth embodim e nt, i n th e mov i ng pictur e cod i ng apparatus i n 
wh i ch the rat e contro l i s conduct e d from th e pr e v i ously cod e d moving pictur e 
data w i thout d e coding tho data and th e mov i ng pictur e data is now l y produced, 
tho mov i ng p i cture cod i ng apparatus charact e r i z e d in that th e i nput mov i ng 
p i cture data has, as th e rat e corr e ction data, th e structur e hav i ng th e I fr a m e 
wh i ch is th e cod e d i mag e in tho fram e whos e b i t amount i s d i ffer e nt to th e norma l 
fram e , w i l l be descr i bed b el ow. I n i t i a ll y, the mov i ng p i cture data produc i ng 
apparatus i n wh i ch th e moving p i cture data which i s an i nput of th e mov i ng 
picture coding apparatus, i s produced, w il l b e d e scr i b e d, and next, th e moving 
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p i cture cod i ng apparatus by wh i ch, from th e mov i ng p i ctur e data cod e d by th e 
moving p i cture data produc i ng apparatus, th e rat e contro l i s conduct e d w i thout 
decod i ng th e data, and th e mov i ng pictur e data i s n e w l y produc e d, w ill b e 
descr i bed bo l ow. The sixth embodiment will be described below. It includes a 
moving picture coding apparatus for performing rate control without decoding on 
previously coded moving picture data, and for producing new moving picture 
data. The moving picture coding apparatus has moving picture data as an input. 
The input moving picture data is structured so that the l-frame, which is the 
coded image in the frame whose bit amount is different to the normal frame, is 
the rate correction data. The moving picture data producing apparatus and the 
moving picture coding apparatus will be described below. 

[0167] The moving picture data producing apparatus produces moving picture 
data, which is inputted to the moving picture coding apparatus. In FIG. 8, the 
structure of the moving picture data producing apparatus 801 to produce the 
input mov i ng p i ctur e data of the mov i ng p i cture cod i ng apparatus is shown. The 
mov i ng p i cture data produc i ng apparatus 801 i s provid e d w i th lt includes the 
following: a frame input means 808 connected to the input means 802[[,]]i_a P z 
frame coding means 803[[, and]]j_a bit amount buffer 804 , and i s provid e d with a 
fram e i nput m e ans 808, and ; an l-frame coding means 805 connected to the bit 
amount buffer 804 . and further, i s prov i ded with a , frame input means 808, and 
compression frame data combination means 809 : the compression frame data 
combination means 809 is connected to the P z frame coding means 803-aBd , the 
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Mrame coding means 805, and a mov i ng pictur e data comb i nat i on m ea ns 806, 
and has th e structur e conn e ct e d to an output means 807. 

[0168] The operation of the moving picture data producing means structur e d 
as descr i b e d abov e 801 will be described below. The frame input means 808 
inputs the non - compress i on non-compressed image into the P-frame coding 
means 803 and Mrame coding means 805 for each [[1] 1single frame. The P z 
frame coding means conducts the P-frame coding through the motion 
compensation, DCT conversion, quantization, and variable length coding 
processing, and outputs the coded frame to the compression frame data 
combination means 809, and outputs the bit amount of the coded frame to the bit 
amount buffer 804. The bit amount buffer 804 outputs the inputted bit amount to 
the Mrame coding means 805. 

[0169] Then, the Mrame coding means 805 conducts the Mrame coding 
through the DCT conversion, quantization, and variable length coding 
processing, on the frame inputted from the input means 802, and outputs the 
coded frame to the compression frame data combination means 809. H e r ei n, 
tfr eThe detailed structure of the Mrame coding means 805 is shown in FIG. 37. In 
FIG. 37, the Mrame coding means 805 is connected to an input means 3701, 
and is structurod b v includes the following: a DCT conversion means 3702[[,]]; 
DCT coefficient memory 3703[[,]]; quantization means 3704[[,]fc variable length 
coding means 3705[[,]]; and quantization value determination means 3706[[,]]; 
and i s connoctod to an output means 3707. In F I G. 37. tho The input means 3701 
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corresponds to the output of the frame input means 808 in FIG. 8, and the output 
of the bit amount buffer 804, and inputs the non - compross i on non-compressed 
frame into the DCT conversion means 3702[[,]] and toptrts-the bit amount into the 
quantization value determination means 3706. The DCT conversion means 3702 
DCT-converts the inputted non - compr e ssion non-compressed data, and outputs it 
to a DCT coefficient memory 3703. The DCT coefficient memory 3703 stores the 
inputted DCT coefficient in the internal memory, and mput soutputs the DCT 
coefficient into a quantization means 3704. Further, when the DCT coefficient 
signal is inputted from the quantization value determination means 3706, the 
DCT coefficient stored in the internal memory is outputted to the quantization 
means 3704. The quantization means 3704 conducts the quantization by using 
the DCT coefficient inputted from the quant i zat i o DCT coefficient memory[[,]] 
3703 and the quantization value inputted from the quantization value 
determination means 3706 , and outputs it to the variable length coding means 
3705. The variable length coding means 3705 conducts the variable length 
coding on the inputted data, and outputs the variable length coding data and its 
bit amount to the output means 3707 , a nd th e . In addition it outputs the bit 
amount to the quantization value determination means 3706. 

[0170] The quantization value determination means 3706 stores the bit 
amount inputted from the input means 3701 in the internal memory, and outputs 
the previously determined quantization value to the quantization means 3704. 
Further, when the bit amount is inputted from the variable length coding means 
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3705, it is compared to the bit amount stored in the internal memory, and the 
next quantization value is determined so that the bit amount becomes small , and 
th e. The quantization value is outputt e d determination means 3706 outputs the 
quantization value to the quantization means 3704[[,]] and the DCT coefficient 
output signal i s outputted to the DCT coefficient memory 3703. Further, by using 
the bit amount inputted from the variable length coding means 3705, the internal 
memory is r e n e w e d updated . 

[0171] After the above processi ngs ar e is repeated [[by]]for the number of 
times rfofll as there are pieces of the rate correction data which is previously 
determined by the user, the quantization value determination means 3706 
outputs the l-frame coding end signal to the output means 3707. 

[0172] As described above, the l-frame coding means 805 produces a 
plurality of l-frames whose bit amount is different from the inputted bit amount, 
and outputs the produced l-frame and its bit amount to the compression frame 
data combination means 809 . Finally, the compression frame data combination 
means 809 combines the P 2 frame data inputted from the P^frame coding means 
803, and the rate correction data produced from a plurality of l-frame data and 
their bit amounts inputted from the l z frame coding means 805, in order, and 
outputs it to the output means 807. FIG. 27 is a view showing the rate correction 
data structure and the rate correction data header structure in the present sixth 
embodiment. Further, FIG. 28 is a view showing the data content of the rate 
correction data shown in FIG. 27. Th e rat e corr e ct i on data i s, as As shown in FIG. 
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27 and FIG. 28, the rate correction data is comprised o f structur e d bv the rate 
correction data header and a plurality of l z frames. The rate correction data 
header has th e structur e in wh i ch is structured such that the number of l-frames 
and respective bit amounts are accommodated in the fixed length. 

[0173] Next, the structure of the moving picture coding apparatus is shown in 
Fig. 9. in wh i ch th e mov i ng picture data cod e d by th e mov i ng picturo data 
producing apparatus 801 is an i nput, and the rato contro l i s conducted w i thout 
cod i ng, and th e mov i ng p i cturo data i s n e wly produc e d, i s shown i n F I G. 9. The 
moving picture coding apparatus 901 has th e structur e wh i ch is prov i d e d 
with includes the following: a rate correction data extraction means connected to 
an input means 902-af*d; a bit rate control means 905 connected to the input 
means 902 and the rate correction data extraction means , and furth e r,^ a rate 
correction means 906 connected to [[a]]the bit rate control means 905, and the 
rate correction data extraction means 903 . and i s connocted to ; and an output 
means 907. 

[0174] The operation of the moving picture coding apparatus structur e d as 
d e scr i b e d abov e , 901 will be described below. The input means 902 inputs the 
moving picture data coded by the moving picture data producing means 801 for 
each rMll single frame[[,]] into the rate correction data extraction means 903 , and ,. 
It also inputs the ob i ectiv e desired bit rate into the bit rate control means 905. The 
rate correction data extraction means 903 extracts the rate correction data from 
the inputted data and outputs it to the bit rate control means 905 , and . It also 
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outputs the normal P z frame data oth e r than th e rat e corr e ct i on data to (excluding 
the rate correction data) to the rate correction means 90&r^ari d. It further outputs 
the bit amount of the P z frame to the bit rate control means 905. 

[0175] Then, the bit rate control means 905 compares the obiectiv e desired bit 
rate inputted from the input means 902, to the bit amount inputted from the rate 
correction data extraction means 903 , and wh e n . When the bit rate is satisfied, 
and the immediately preceding f rame wh i ch i s preced i ng 1 fram e is not frame- 
skipped, the control signal in which the rate correction is not necessary[[,]] is 
outputted to the rate correction means 906. In contrast to that, when the bit rate 
is not satisfied, or the immediately preceding frame which is pr e c e d i ng 1 fram e is 
frame-skipped, the rate correction data header is referred to, and from ^frames m 
tho rato correct i on data, tho I framo which satisfies the ob i oct i vo desired bit rate is 
selected , and th e . The selected l-frame is then outputted to the rate correction 
means [[70611 906 . Further, even when the rate correction data is used, when the 
bit amount is excessive, the frame skip control signal is outputted to the rate 
correction means 906, and the information whether the frame skip control signal 
is emitted[[,]] is stored in the internal memory. 

[0176] Furth e r, wh e n th e contro l s i gnal of tho rat e corr e ct i on unn e c e ssary is 
i nputt e d from tho b i t rato contro l moans 905, tho rato corr e ct i on m e ans 906 
outputs tho fram e i nputt e d from th e rat e corr e ction data extraction m e ans 903 as 
it i s, to tho output moans 907, and whon tho I framo i s i nputted from th e b i t rate 
control means 905, th e I fram e is outputt e d to tho output m e ans 907, and wh e n 
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th e fram e sk i p contro l signal i s i nputt e d from th e b i t rat e contro l m e ans 905, th e 
fram e sk i p contro l s i gnal is outputt e d to th e output m e ans 9 07. F i nal l y, the output 
m e ans 907 coll e cts th e fr a m e dat a i nputt e d from th e rat e corr e ct i on m e ans 906 
for e ach 1 fram e , and fr a m e sk i p control signa l , and th e moving picturo data i s 
produc e d. I n this conn e ct i on, when th e fram e skip control signal i s i nputted, th e 
frame is sk i pp e d. Further, when the control signal in which the rate correction is 
not necessary is inputted from the bit rate control means 905. the rate correction 
means 906 outputs, to the output means 907, the frame inputted from the rate 
correction data extraction means 903 unchanged. When the l-frame is inputted 
from the bit rate control means 905, the l-frame is outputted to the output means 
907. When the frame skip control signal is inputted from the bit rate control 
means 905, the frame skip control signal is outputted to the output means 907. 
Finally, the output means 907 collects the frame data inputted from the rate 
correction means 906 for each frame, and frame skip control signal, and the 
moving picture data is produced. In this connection, when the frame skip control 
signal is inputted, the frame is skipped. 

[0177] In the present embodiment, bec a u s e th e i nput moving p i ctur e data has 
th e I fram e a s the rat e corr e ct i on data the generation of motion compensation 
error can be prevented , even when the frame skip is conducted at the time of 
rate control, by using the l-frame for the next frame , th e g e n e ration of th e mot i on 
comp e nsat i on e rror can b e prevented . This is because the input moving picture 
data has the l-frame as the rate correction data. 
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[0178] Further, the number of pieces of the l-frame which is the rate 
correction data, and the quantization value when the rate correction data is 
produced, can be set by the user corresponding to the range of the bit rate 
change. FeeFor example, when the quantization of the normal frame data is 
conducted at the quantization value Q=2, and the bit rate is about 1.6 Mbps, the 
quantization is conducted by using 2 quantization values of Q=6, Q=29, as the 
rate correction data, and the data of about 384 kbps and about 64 kbps are 
produced. Then, by using together the rate correction data or frame skip, it is 
possible to produce -the moving picture data i n wh i ch having the arbitrary bit rate 
change of range from about 1.6 Mbps to about 64 kbps i s poss i bl e , can b e 

[0179] As described above, in the present embodiment, when the moving 
picture data producing apparatus has a means for producing the Mrame whose 
bit amount is different[[,]] as the rate correction data, and the moving picture 
coding apparatus is provided with , as th e rat e corr e ct i on data, a means for 
selecting the l-frame as the rate correction data and conducting the rate control, 
the rate control can be quickly conducted without decoding the input moving 
picture data , and furthe r . Further , the generation of the-motion compensation 
error when the frame-skip is conducted[[,]] can be prevented, and a plurality of 
moving picture data whose bit rates are different can be quickly produced , and i ts 
pract i ca l e ff e ct i s l argo . 

[0180] Embodiment 7 
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[0181] In the seventh embodiment, in — an — apparatus — by — which — the 
fat edescribed below, rate control is conduced from th eon previously coded 
moving picture data without decoding the data[[,]] and thenew moving picture 
data is newty-producedff.il . This embodiment further includes a moving picture 
coding apparatus characterized in that a portion of the area in the frame of the 
moving picture data is selected and removed auarri e d out , and the frame size is 
different from the input moving picture data , and th e . The moving picture data^ 
whose size is the quarr i od ou t selected and removed frame size, is producedff,]] 
and will be described below. When the size is quarr ie d ou t selected and removed 
from the frame size of the input image data and the moving picture data is 
produced, b e caus e the bit rate is not un i form, th e b i t i s not regulated depending 
on the quarr ie d out selected and removed portion because the bit rate is not 
uniform , and in order to quarr ie d ou t select and remove data , it is necessary that 
the bit rate [[is]]be changed. 

[0182] I n i t i al l y, th e mov i ng p i ctur e data produc i ng apparatus by wh i ch tho 
mov i ng p i cture data which i s an i nput of tho moving p i cture cod i ng apparatus is 
produc e d, w ill b e d e scr i b e d, and n e xt, th e moving p i ctur e cod i ng apparatus by 
which th e rat e contro l i s conduct e d from th e pr e vious l y cod e d moving p i ctur e 
data w i thout d e cod i ng th e data, and th e mov i ng pictur e data i s n e wly produc e d, 
w ill bo descr i b e d b el ow. lnitiallv. the moving picture data producing apparatus and 
moving picture coding apparatus will be described below. The moving picture 
data producing apparatus produces moving picture data, which is an input to the 
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moving picture coding apparatus. The moving picture coding apparatus performs 
rate control without decoding the input data and produces new moving picture 
data. 

[0183] In FIG. 29, the structure of the moving picture data producing 
apparatus to produc e th e i nput mov i ng p i ctur e data of th e mov i ng p i ctur e coding 
apparatus 2901 is shown. I n F I G. 29, a The moving picture data producing 
apparatus 2901 is provided with: a frame input means 117 connected to an input 
means 116; motion compensation means 2902; DCT conversion means 103; 
quantization means 104; variable length coding means 105; inverse quantization 
means 106 to conduct the decoding; inverse DCT conversion means 107; and 
frame memory 108 to store the decoded fram e, and i s prov i d e d with: ; a 
maximum bit amount area detecting means 2910 to detect the area having the 
maximum bit amount, connected to the variable length coding means 105; 
reference inhibition area memory means 109; compression frame buffer 112 
connected to the DCT conversion means 103; quantization means 2911 to 
conduct the quantization, connected to the compression frame buffer 112; ao4 
variable length coding means 2913, and i s prov i d e d w i th: a compression frame 
data combination means 2914 to combine the moving picture data, connected to 
the variable length coding means 105, reference inhibition area memory means 
109, af^d-variable length coding means 2913 , and has th e structur e conn e ct e d to; 
and the output means 115. The operation of the moving picture data producing 
apparatus structured as doscr i bod abovo 2901 will be described below. In FIG. 
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29, operations of the motion compensation means 2902, maximum bit amount 
area detecting means 2910, quantization means 2911, variable length coding 
means 2913, and the operation of the blocks other than compression frame data 
combination means 2914 are entirely the same as in the Embodiment 1. 

[0184] An example of the quarry ou t select and remove area is shown in FIG. 

30. In FIG. 30, [[111 a single frame is divided into 4 quarry ou t select and remove 
areas surrounded by a bold line. The moving picture data produced by the 
moving picture data producing apparatus 2901 m e ans i n th e pr e s e nt ombodiment 
is different form the input frame size, and has the structure in which, for example, 
the moving picture data can be produced by the moving picture coding 
apparatus, which will be described later, as the frame size of an arbitrary select 
and remove area ( quarry out area} shown in FIG. 30. 

[0185] In the Embodiment 1, the maximum bit amount area detecting means 
110 selects a plurality of areas in the order of bit amount from the rang e inc l uding 
the-maximum b i t amount in the plurality of areas in [f11] a single frame shown in 
FIG. 10 , and to th e s ele ct e d ar e a, b y . By changing the quantization value for the 
selected area by the quantization means 111, the rate correction data whose bit 
amount is different is produced. 

[0186] In contrast to that, in the present embodiment, the maximum bit 
amount area detecting means 2910 selects, for example, as shown in FIG. 30, te 
4for a single frame which is divided into a plurality of select and remove areas 
(quarry out areas (rectangu l ar area shown by a bo l d lin e) that are shown outlined 
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in bold , a plurality of areas in the order from the area in which the bit amount is 
maximum in each quarry ou t select and remove out area , and outputs th e . Each 
selected eachout area is outputted to the reference inhibition area memory 
means 109, and to the compression frame buffer 112. In this connection, the 
quarr i ed ou t selected and removed area shown in FIG. 30 is an example[[,]] and 
tho quarry out ar e a can be arbitrarily determined. Further, the motion 
compensation means 2902 conducts the motion compensation by inhibiting the 
motion estimation to the reference inhibition area shown in the rate correction 
data 34— inputted from the reference inhibition area memory means and the 
quarry ou t select and remove area of the position which is different from the 
currently coded q^aff vselect and remove area. For example, when the motion 
compensation is conducted on th e quarry ou t select and remove area 1, the 
motion estimation is conducted only from the area excepting fo r other than the 
reference inhibition area in the quarry ou t select and remove area 1, in one 
preceding frame to the frame into which the input is conducted from the frame 
memory 108. Tentatively, when the motion estimation is conducted to th e for the 
area outside of the quafr yselect and remove area, decoding cannot be 
conducted because , although moving picture data is produced, there is no 
reference image to be used for the motion compensation when the area is 
quarri e d selected and removed from the inside of one fram o and tho mov i ng 
picture data i s produced, tho decod i ng can not be conducted . As described 
above, by providing the limitation for the motion estimation, the decoding of the 
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coded moving picture data can be conducted by using not only the whole frame, 
but also only each select and remove ( quarr y out) area , and tho . Furthermore, 
the moving picture data can be structured by quarry i ng out selecting and 
removing a portion from the frame of the coded moving picture data. 

[0187] The quantization means 2911 changes the quantization value and 
conducts the guantization by us i ng . For this is uses the DCT coefficient and 
quantization value of each area selected by the reference inhibition area shown 
in the rate correction area data inputted from the compression frame buffer 112, 
that is, selected by the maximum bit amount area detecting means 2910 , and 
produc e s th e data . The quantization means 2911 produces the data whose bit 
amount is different, and respectively outputs to the variable length coding means 
2913. 

[0188] The variable length coding means 2913 conducts the variable length 
coding on the DCT coefficient whose bit amount is different[[, ofll for each 
reference inhibition area inputted by the quantization means 291 1[[, and]1 . The 
variable length coding means 2913 produces the rate correction data having 
respective data sizes and area numbers as the header information, and outputs it 
to the compression frame data combination means 114. Herein, the data 
produced by the variable length coding means 2913 is called the rate correction 
data. In FIG. 33, the structure of the rate correction data is shown. In FIG. 33, the 
rate correction data header 3302, as shown in FIG. 34, has the structure in which 
the number of the correction data in each area, the area number in the frame, 
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and the bit amounts of respective correction data, are accommodated as the 
fixed length data. Then, the rate correction data has th eis structured in which so 
that the correction data is accommodated in order succ ee ding to followinq the 
rate correction data header 3302. 

[0189] The compression frame data combination means 2914 links in order 
the normal frame data inputted by the variable length coding means 105, the rate 
correction area data inputted by the reference inhibition area memory means 
109, and the rate correction data inputted by the variable length coding means 
2913 as shown in FIG. 32 , and . It produces the compression frame data, 
af ^which it outputs [[it]] to the output means 115. 

[0190] Next, [[in]] FIG. 35[[,]]_shows the structure of the moving picture coding 
apparatus 3501 by which a portion in the frame is quarr ie d selected and removed 
from the moving picture data cod e d by th e mov i ng p i ctur e cod i ng apparatus in 
F I G. 29 w i thout b e ing coded , and the bit rate adjustment following the quarrying 
eu tselection and removal is conducted, and the new moving picture data is n e wly 
prod uced , is shown . 

[0191] In FIG. 35, the moving picture coding apparatus 3501 has the structure 
which is provided with: a data separation means 3507 connected to an input 
means 3502; bit amount calculation means 2303 203 ; rate correction data 
selection means 2404 204 ; thee bit rate control means 205; and moving picture 
data combination means 208, and which is connected to the output means 206. 
The operation of the moving picture coding apparatus 3501 structure as 
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d e scr i b e d a bov e will be described below. In FIG. 35, the operations of blocks 
other than the input means 3502 and data separation means 3507, are e nt i r el y 
the same as in Embodiment 1. 

[0192] Th e i nput means 3502 inputs th e mov i ng p i ctur e data wh i ch i s 
compr e ss i on cod e d by th e mov i ng p i ctur e data produc i ng moans 2901, objoct i vo 
bit rate, quarry ar e a i nformat i on show i ng th e quarry out method of 1 frame as 
shown i n F I G. 30, and quarry out ar e a numb e r show i ng wh i ch portion of th e i nput 
data is to bo quarr i od out, to tho data separation m e ans 207. Wh e n th e data i s 
i nputt e d i nto th e data s e par a tion m e ans 3507 by th e i nput m e ans 3502, th e 
obj e ct i v e b i t rat e i s i nputt e d i nto th e b i t rat e contro l m e ans 205 for oach 1 framo, 
and tho data is takon out i n ordor from th e le ading e nd of th e i nputt e d mov i ng 
pictur e data, th e quarry i ng out and composition i s conduct e d on tho norma l fram e 
data corr e sponding to th e quarry out area numb e r, and i t is i nputted into tho bit 
amount ca l cu l at i on m e ans 203 for oach 1 fram e , and tho rato correction data 
corr e spond i ng to tho rat e corr e ction ar e a data and th e quarry out ar e a numb e r is 
i nputted i nto tho rato correct i on data s o l o ct i on moans 204. The input means 3502 
inputs to the data separation means 3507 the moving picture data which is 
compression coded by the moving picture data producing means 2901, desired 
bit rate, select and remove (quarry out) area information showing the select and 
remove method for a single frame as shown in FIG. 30, and select and remove 
(guarry out) area number showing which portion of the input data is to be 
selected and removed. When the data is inputted into the data separation 
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means 3507, the desired bit rate is inputted into the bit rate control means 205 
for each single frame. The data is taken out in order from the leading end of the 
inputted moving picture data and the guarrving out and construction is conducted 
on the normal frame data corresponding to the select and remove area number. 
This modified data is inputted into the bit amount calculation means 203 for each 
single frame, and the rate correction data corresponding to the rate correction 
area data and the select and remove (guarrv out) area number is inputted into 
the rate correction data selection means 204. 

[0193] G e n e r all y, in tho caso whoro, to tho prev i ously coded i mag e data, a 
port i on of th e fram e i s quarr i od out and tho now moving picture data whos e 
fram e siz e i s diff e rent, is produc e d, wh e n th e mot i on e st i mat i on is conduct e d 
from th e ar e a oth e r than th e ar e a i n which th e pr e v i ously cod e d i mag e data i s 
quarr ie d out, th e prob le m i n which i t can not b e d e cod e d, i s g e n e rat e d. That i s for 
th e r e ason why the n e cessary r e f e r e nc e data to d e cod e th e quarr i od out ar e a 
does not oxist i nsido tho quarriod out aroa. That i s, bocauso tho necessary 
r e f e rence data e x i sts outs i d e th e quarri e d out ar e a, i t i s i mpossib le that th e 
quarry i ng out i s s i mply conduct e d, and i n ord e r to quarry out a portion of th e 
fram e and produc e th e n e w mov i ng p i ctur e data whoso fram e siz e i s d i ff e r e nt, i t 
i s n e c e ssary that, aft e r who le fram e is d e cod e d onc e , tho i ns i d e of th e quarry out 
aroa i s codod aga i n, and thoro i s a prob l em that tho process i ng l oad i s 
tefger Generallv. when previously coded image data is provided, and a portion of 
the frame is selected and removed, producing new moving picture data whose 
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frame size is different, motion estimation conducted from the area other than the 
area in which the previously coded image data is selected and removed is 
problematic because the data cannot be decoded. That is the reason for why the 
necessary reference data to decode the selected and removed area does not 
exist inside the selected and removed area. That is, because the necessary 
reference data exists outside the selected and removed area, it is not possible 
that the selection and removal is simply conducted. In order to select and 
remove a portion of the frame and produce the new moving picture data whose 
frame size is different, it is necessary that, after the whole frame is decoded 
once, the inside of the select and remove area is coded again. This extra coding 
is problematic in that the processing load is large. 

[0194] In contrast to that, in the present embodiment, when the data 
separation means 3507 guarri e s out selects and removes the data of the area 
shown by the quarry ou t select and remove area number from the normal frame 
data, and composes the new frame whose frame size is different, because the 
motion estimation of the guarr ie d selected and removed area is conducted only in 
the same area, there is no case that the motion compensation error is generated 
by the quarrying [foutll and removing , and it can not be decoded. Accordingly, it 
can be guarr i od out selected and removed and composed without decoding the 
data. Further, a l so the change of bit rate following the quarrying out selection and 
removal can be conducted without decoding the data by selecting the rate 
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correction data by adjusting to the obi e ct i v e desired bit rate in the same manner 
as in Example 1. 

[0195] Further, the number of areas having the rate correction data and the 
rate correction data of each area.,, can be set by the user corresponding to the 
range of the bit rate change, and has the same effect as in Embodiment 1 7 
howovo r . However , the rate correction data necessary only for conducting the bit 
rate adjustment following the quarry i ng ou t selection and removal may be small. 
For example, when tho quarry ou tt here are four select and remove areas are 4^ 
and the quantization value of the normal frame data Q=8, and the bit rate is 
about 256 kbps[[,]] as the rate correction data, at least one area in which the bit 
amount is large is selected in respective quarry ou t select and remove areas , and 
when . When the rate correction data of about 96 kbps (as the whole frame) in 
which the quantization is conducted by using the quantization value Q=20[[,]] is 
produced, it is suffici e nt l y e nouqh sufficient enough for adjustment of the bit rate 
generated following the quarry ou t selection and removal . That is, when the 
quarry out ar e as i s 4 there are four select and remove areas , and the bit rate is 
about 256 kbps, it is necessary that the bit rate of the frame quarri e d out by th e 
quarry out of th e ar ea is selected and removed area be 1/4 of the whole, that is, 
about 64 kbps , how e v e r . However , in practice, by the deviation of the bit 
amount, the area over 64 kbps exists. In the present embodiment, the-rate 
correction data is produced , and b y . By using the rate correction data at the time 
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of quarry out selection and removal of the area, the bit rate of such thean area 
can be adjusted. 

[0196] As described above, in the present embodiment^,! ! includes the 
moving picture data structure having th e with an area having the rate correction 
data in which the bit amount is different for each quare vselect and remove area 7 
a. It further includes means by which the area of a portion in the frame is 
quarr ie d out and compos e d selected and removed and constructed , and an area 
selection means for selecting the rate correction data corresponding to the bit 
rat e, ar e prov i d e d, th e r e b y . Thereby , a portion of the frame is quickly quarr ie d 
eut selected and removed without decoding the moving picture data[[,]] and 
without generating the deterioration of image qualit y, and furthe r . Further , the bit 
rate adjustment can be conducted and the new moving picture data whose frame 
size is different, can be produced , and its pract i ca l offoct i s l argo . 

[0197] Further, in the present embodiment, when a po i nt that th ean area 
having the reference inhibition area and rate correction data is produced 4f>-4 for a 
single frame[[,]] is changed [[to]]so that it is produced for each quarry ou t select 
and remove areas, and following that, also for the motion compensation, the 
motion estimation is not conducted from th e outside of the quarry ou t select and 
remove area , i t is . It has been shown that tho offoct i n which, not on l y by 
conduct i ng th e rat e contro l without d e coding tho cod e d moving p i ctur e data, but 
a l so by quarrying out a port i on in tho framo w i thout decod i ng tho data, tho 
moving pictur e data whos e framo s i z e i s d i fferent can b e produc e d, can b e 
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obtain e d, and fo r movinq picture data whose frame size is different can be 
produced not only by conducting the rate control without decoding the coded 
moving picture data, but also by selecting and removing a portion in the frame 
without decoding the data. For Embodiment 2 to Embodiment 6, by conducting 
the same change, a portion in the frame is guarr i od ou t selected and removed , 
and the moving picture data can be produced. 

[0198] Further, in Embodiments 1 to 7, i t is d e scribed that the moving picture 
data producing apparatus accommodatos stores the rate correction data after the 

normal frame dat e , — how e v e r data. However , the position to — be 

accommodatod, where the correction data is stored is not limited to this. For 
example, when the moving picture data producing apparatus 
accommodat e s stores the rate correction data after the user data start sign with in 
the moving picture data, the data can be reproduced by using the normal moving 
picture decoding apparatus. 

[0199] Herein, the user data start sign is, for example, as shown in I SO/I EC 
11172 - 2 wh i ch is th e standard book of MPEG cod i ng MPEG coding standard 
ISO/IEC 11172-2 , a sign showing the start of the area which is prepared for the 
future expansion, and the normal moving picture decoding apparatus skips over 
the data from the user data start sign to the next start sign and conducts the 
decoding. Accordingly, the moving picture data produced by the moving picture 
data producing apparatus in Embodiments 1 to 7, can be reproduced by using 
the normal moving picture decoding apparatus. 
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[0200] Further, in the present invention, the input data of the moving picture 
coding apparatus and output data are compared with each other, or a plurality of 
output data whose bit rates are different[[,]] are compared , and wh e n . When the 
bit arrangement is locally different, it is structured in such a manner that the bit 
arrangement is accommodat e d stored in the user data, or a_position other than 
the normal frame data. 

[0201] Further, when the monochrome frame is an input, each frame of the 
moving picture data outputted by the moving picture data producing apparatus 
has the periodic structure in which, other than normal frame data, as the rate 
correction data, the e nt i r el y same data are periodically included by tho for as 
many as the number of areas between the area s, and b e caus e . Because the 
entir el y th e same rate correction data is also included between the frames, the 
moving picture data has the periodic structure. 

[0202] As d e scr i b e d abov e , first l y, when th e mov i ng p i ctur e data has th e 
structur e hav i ng tho rato corr e ct i on data whos e b i t amount is different, to tho 
area i n wh i ch the b i t amount in tho P framo (prod i ct i vo cod i ng i mag e b e tw ee n 
fram e s) i s l arg e , i n th e cas e wh e r e th e moving pictur e data is n e wly produc e d 
from th e pr e v i ous l y cod e d mov i ng pictur e data, by s ele cting th e data whos e b i t 
amount is differ e nt, correspond i ng to th e obj e ctiv e b i t rato, tho bit rat e can b e 
chang e d without d e cod i ng tho mov i ng p i ctur e data, and tho mov i ng p i cture data 
can b e qu i ck l y producod. As described above, first, when new moving picture 
data is produced from previously coded data, and the new data is structured so 
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that it includes the rate correction data whose bit amount is different for the 
areas in which the bit amount in the P-frame (predictive coding image between 
frames) is large, by selecting the data whose bit amount is different, 
corresponding to the desired bit rate, the bit rate can be changed without 
decoding the moving picture data. Accordingly, the moving picture data can be 
quickly produced. 

[0203] S e cond l v Second , in the case where the moving picture data 
fras includes the rate correction data in which the bit amount is different, and the 
rate change can be conducted , on tho prodotorminod area i n which th e 
probab ili ty r o f o rr o d to at tho t i me of th e mot i on e st i mation from th e n e xt fram e is 
lew in the P-frame (predictive coding image between frames), for the 
predetermined area in which the likelihood of being referred to from the next 
frame at the time of the motion estimation is low, and the new moving picture data 
is fiewly-produced from the previously coded moving picture data, by selecting 
on e data from the rate correction data whose bit amount is different, with in the 
input moving picture data, corresponding to the ob ie ct i v e desired bit rate, the bit 
rate can be changed without decoding the moving picture data-; — aftd.. 
Accordingly, the moving picture data can be quickly produced. Further, by 
producing the rate correction data in the area in which the probabil i t v likelihood of 
being referred to from the next frame is low, the l ow e ring reduction of the 
predictive coding efficiency due to the influence of the search area limitation of 
the motion estimation^,]] can be reduced. 
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[0204] Third l v Third , in the moving picture data producing apparatus, by 
providing the motion compensation means for inhibiting the reference, at the time 
of the motion estimation of the next frame, [[to]]for the area having the rate 
correction data in the P^frame of the moving picture data to conduct the coding, 
when the new moving picture data is newly-produced from the coded moving 
picture data, even by selecting the rate correction data whose bit amount is 
different, corresponding to the objective bit rate, the generation of the motion 
compensation error due to the change of the data can be prevented. 

[0205] Fourth l y Fourth , in the moving picture data producing apparatus, when 
the P z frame of the moving picture data is produced, [[to]]for the area in which the 
referred degree data showing the area in the preceding frame used for reference 
at the time of motion estimation is recorded, and by using the referred area data, 
which is selected as the area that the referred degree is low, by providing the 
means for producing the rate correction data whose bit amount is different, the 
moving picture coding can be conducted without low e r i ng decreasing the coding 
efficiency of the predictive coding between frames. Further, when [fthell new 
moving picture data is newly^produced from the previously coded moving picture 
data, by selecting the data whose bit amount is different, corresponding to the 



the data , and the mov i ng pictur e data can b e produc e d . 

[0206] F i fthly Fifth , in the moving picture data producing apparatus, by 
providing the means for producing the data in which the predictive coding 




esired bit rate, the bit rate can be quickly changed without decoding 
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between frames is conducted on the original image to the P-frame of the moving 
picture data, and the means by which the high frequency component of the 
original image is removed, and by conducting the predictive coding between 
frames, the rate correction data whose bit amount is different[[,]] is produced T 
when . When the new moving picture data is fiewly^produced without decoding 
the coded moving picture data, by selecting a plurality of areas in the rate 
correction data, in which the high frequency component is removed and coded, 
and bit amounts are different, the fine bit rate control can be quickly conducted. 

[0207] SktblySixth, in the moving picture data producing apparatus, by 
providing the means for producing the position at which the subsequent bit can 
be deleted, and the termination data to each V i d e o Pack e t video packet of the 
moving picture dataff.fl : and in the moving picture coding apparatus by which the 
new moving picture data is produced from the coded moving picture data-is 
produced , by selecting the V i d e o Pack e t corr e spond i nq video packet according to 
the ob ie ct i v e desired bit rate, and by deleting the rear portion bits, the bit rate 
control can be quickly conducted. 

[0208] S e v e nth l y, — ^ Seventh, when the moving picture data producing 
apparatus , i n th e cas e wh e r e includes the means for producing a plurality of I- 
frames whose bit amounts are different[[,]] as the rate correction data, to the P z 
frame of the moving picture data, is prov i ded, whon tho and new moving picture 
data is n e w l y produced from the previously coded moving picture data, by 
selecting or frame skipping the rate correction data corresponding to the 
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objective bit rate, the-bit rate control can be quickly conducted, and the moving 
picture data can be produced. 

[0209] E i ghth l y, in th e cas e wh e r e th e structur e in wh i ch an ar e a hav i ng 
tog e th e r th e rat e corr e ct i on data whos e b i t amount is d i ff e r e nt i n th e P fram e 
(pr e d i ct i ve cod i ng i mag e betw ee n frames) of th e moving p i cture data i s provided 
at le ast on e for e ach quarry out ar e a d e t e rm i n e d i n th e fram e i s provid e d, and th e 
moving picture dat a produc i ng m e ans i s provid e d w i th th e motion comp e nsat i on 
m e ans i n wh i ch th e motion e st i mation i s not conduct e d from tho different quarry 
out ar e a and the area having th e rate corr e ct i on data, whon ono port i on i n tho 
framo i s quarr i ed out from the prev i ously cod e d mov i ng p i ctur e data and th e 
moving p i ctur e data i s n e w l y produc e d, by sel e ct i ng the data whos e b i t amount i s 
d i ff e r e nt, corr e spond i ng to th e obj e ct i v e b i t rat e , th e b i t rat e can b e contro lle d 
without d e cod i ng th e mov i ng pictur e data, and w i thout g e n e rating th e mot i on 
comp e nsat i on e rror, and th e mov i ng p i cture data can b e quick l y produc e d. 
Eighth, when a current embodiment includes structure wherein at least one area 
having the rate correction data whose bit amount is different in the P-frame of the 
moving picture data is provided for each select and remove area (the select and 
remove area is determined in the frame); and the moving picture data producing 
means is provided with the motion compensation means in which the motion 
estimation is not conducted from the different select and remove area and the 
area having the rate correction data, when one portion in the frame is selected 
and removed from the previously coded moving picture data and new moving 

Substitute Specification, Marked-up copy Page 90 of 91 



Serial No.: 09/899,907 
Filed: July 5, 2001 
Conf.No.:3598 
Atty. Docket No.: 33782 



picture data is produced; by selecting the data whose bit amount is different, 
corresponding to the desired bit rate, the bit rate can be controlled without 
decoding the moving picture data, and without generating the motion 
compensation error, and the moving picture data can be quickly produced. 
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